Skip to content

Commit

Permalink
Merge branch 'main' into formatting-cleanup
Browse files Browse the repository at this point in the history
  • Loading branch information
bmorelli25 authored Feb 4, 2025
2 parents fd41612 + b5beadb commit cd5369e
Show file tree
Hide file tree
Showing 29 changed files with 240 additions and 57 deletions.
4 changes: 2 additions & 2 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/.github/ @elastic/doc-leads
/.github/workflows/co-docs-builder.yml @elastic/docs-engineering
/.github/workflows/ @elastic/docs-engineering

/serverless/ @elastic/platform-docs
/serverless/serverless-changelog.asciidoc @elastic/docs
/serverless/serverless-changelog.asciidoc @elastic/docs
13 changes: 13 additions & 0 deletions .github/workflows/docs-build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
name: docs-build

on:
pull_request: ~

jobs:
preview:
uses: elastic/docs-builder/.github/workflows/preview-build.yml@main
with:
continue-on-error: true
strict: false
permissions:
contents: read
6 changes: 3 additions & 3 deletions .github/workflows/docs-cleanup.yml
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
name: docs
name: docs-cleanup

on:
pull_request_target:
types:
- closed

jobs:
docs-preview:
preview:
uses: elastic/docs-builder/.github/workflows/preview-cleanup.yml@main
permissions:
contents: read
contents: none
id-token: write
deployments: write
16 changes: 16 additions & 0 deletions .github/workflows/docs-deploy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
name: docs-deploy

on:
workflow_run:
workflows: [docs-build]
types:
- completed

jobs:
preview:
uses: elastic/docs-builder/.github/workflows/preview-deploy.yml@main
permissions:
contents: none
id-token: write
deployments: write
actions: read
25 changes: 0 additions & 25 deletions .github/workflows/docs.yml

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -424,5 +424,5 @@ The `constant_keyword` is not strictly required for this optimization: it is als

## Default search timeout [_default_search_timeout]

By default, search requests don’t time out. You can set a timeout using the [`search.default_search_timeout`](../../../solutions/search/querying-for-search-searching-with-the-search-api.md#search-timeout) setting.
By default, search requests don’t time out. You can set a timeout using the [`search.default_search_timeout`](../../../solutions/search/querying-for-search.md#search-timeout) setting.

2 changes: 1 addition & 1 deletion explore-analyze/aggregations.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ An aggregation summarizes your data as metrics, statistics, or other analytics.

## Run an aggregation [run-an-agg]

You can run aggregations as part of a [search](../solutions/search/querying-for-search-searching-with-the-search-api.md) by specifying the [search API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html)'s `aggs` parameter. The following search runs a [terms aggregation](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html) on `my-field`:
You can run aggregations as part of a [search](../solutions/search/querying-for-search.md) by specifying the [search API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html)'s `aggs` parameter. The following search runs a [terms aggregation](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html) on `my-field`:

```console
GET /my-index-000001/_search
Expand Down
36 changes: 35 additions & 1 deletion explore-analyze/ai-assistant.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,38 @@ mapped_urls:
% - [ ] ./raw-migrated-files/docs-content/serverless/security-ai-assistant.md
% - [ ] ./raw-migrated-files/docs-content/serverless/ai-assistant-knowledge-base.md

$$$token-limits$$$
$$$token-limits$$$

**AI Assistant** is a chat-based interactive tool that uses generative AI and ELSER, Elastic’s proprietary semantic search model, to help you with a variety of tasks related to Elasticsearch and Kibana, including:

- **Constructing queries**: Assists you in building queries to search and analyze your data, including converting queries from other languages to [ES|QL](query-filter/languages/esql-rest.md).
- **Indexing data**: Guides you on how to index data into Elasticsearch.
- **Using APIs**: Calls Elasticsearch APIs on your behalf if you need specific operations performed.
- **Generating sample data**: Helps you create sample data for testing and development purposes.
- **Visualizing and analyzing data**: Assists you in creating visualizations and analyzing your data using Kibana.
- **Troubleshooting**: Explains errors, messages, and suggests remediation.

AI Assistant requires specific privileges and a generative AI connector.

% Check [Configure AI Assistant](../deploy-manage/) for more details on how to enable and configure it.

The capabilities and ways to interact with AI Assistant can differ for each solution. Find more information in the respective solution docs:

% - [AI Assistant for Search](../solutions/search/)
- [AI Assistant for Observability](../solutions/observability/observability-ai-assistant.md)
- [AI Assistant for Security](../solutions/security/ai/ai-assistant.md)

## Prompt best practices [rag-for-esql]
Elastic AI Assistant allows you to take full advantage of the Elastic platform to improve your operations. It can help you write an ES|QL query for a particular use case, or answer general questions about how to use the platform. Its ability to assist you depends on the specificity and detail of your questions. The more context and detail you provide, the more tailored and useful its responses will be.

To maximize its usefulness, consider using more detailed prompts or asking for additional information. For instance, after asking for an ES|QL query example, you could ask a follow-up question like, “Could you give me some other examples?” You can also ask for clarification or further exposition, for example "Please provide comments explaining the query you just gave."

In addition to practical advice, AI Assistant can offer conceptual advice, tips, and best practices for enhancing your security measures. You can ask it, for example:

- “How do I set up a machine learning job in Elastic Security to detect anomalies in network traffic volume over time?”
- “I need to monitor for unusual file creation patterns that could indicate ransomware activity. How would I construct this query using EQL?”

## Your data and AI Assistant [ai-assistant-data-information]
Elastic does not use customer data for model training. This includes anything you send the model, such as alert or event data, detection rule configurations, queries, and prompts. However, any data you provide to AI Assistant will be processed by the third-party provider you chose when setting up the generative AI connector as part of the assistant setup.

Elastic does not control third-party tools, and assumes no responsibility or liability for their content, operation, or use, nor for any loss or damage that may arise from your using such tools. Please exercise caution when using AI tools with personal, sensitive, or confidential information. Any data you submit may be used by the provider for AI training or other purposes. There is no guarantee that the provider will keep any information you provide secure or confidential. You should familiarize yourself with the privacy practices and terms of use of any generative AI tools prior to use.
25 changes: 24 additions & 1 deletion explore-analyze/alerts.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,4 +15,27 @@ mapped_urls:

$$$alerting-concepts-actions$$$

$$$alerting-concepts-conditions$$$
$$$alerting-concepts-conditions$$$

Alerting tools in Elasticsearch and Kibana provide functionality to monitor data and notify you about significant changes or events in real time. This page provides an overview of how the key components work.

## Alerts
Alerts are notifications generated when specific conditions are met. These notifications are sent to you through channels that you previously set such as email, Slack, webhooks, PagerDuty, and so on. Alerts are created based on rules, which define the criteria for triggering them. Rules monitor the data indexed in Elasticsearch and evaluate conditions on a defined schedule to identify matches. For example, a threshold rule can generate an alert when a value crosses a specific threshold, while a machine learning rule activates an alert when an anomaly detection job identifies an anomaly.

## Cases
Cases are a collaboration and tracking tool, which is particularly useful for incidents or issues that arise from alerts. You can group related alerts into a case for easier management, add notes and comments to provide context, track investigation progress, and assign cases to team members or link them to external systems. Cases ensure that teams have a central place to track and resolve alerts efficiently.

## Maintenance windows
If you have a planned outage, maintenance windows prevent rules from generating notifications in that period. Alerts still occur but their notifications are suppressed.

### Workflow Example

1. **Rule Creation**: You set up a rule to monitor server logs for failed login attempts exceeding 5 within a 10-minute window.
1. **Alert Generation**: When the rule's condition is met, an alert is created.
1. **Notification**: The alert runs an action, such as sending a Slack message or an email, unless a maintenance window is active.
1. **Case Management**: If the alert is part of an ongoing investigation, it's added to a case for further analysis and resolution.

By combining these tools, Elasticsearch and Kibana enable incident response workflows, helping teams to detect, investigate, and resolve issues efficiently.

## Watcher
You can use Watcher for alerting and monitoring specific conditions in your data. It enables you to define rules and take automated actions when certain criteria are met. Watcher is a powerful alerting tool for custom use cases and more complex alerting logic. It allows advanced scripting using Painless to define complex conditions and transformations.
59 changes: 59 additions & 0 deletions explore-analyze/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,3 +16,62 @@ mapped_urls:
% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc):

$$$elasticsearch-explore-your-data-visualizations-save-to-the-visualize-library$$$

The Elasticsearch platform and its UI, also known as Kibana, provide a comprehensive suite of tools to help you search, interact with, explore, and analyze your data effectively. These features empower you to gain deep insights, uncover trends, and take actionable steps based on your findings. This page is an overview of the key capabilities.

## Querying and filtering
Elasticsearch’s robust query capabilities enable you to retrieve specific data from your datasets. Using the Query DSL (Domain Specific Language), you can build powerful, flexible queries that support:

- Full-text search
- Boolean logic
- Fuzzy matching
- Proximity searches
- Semantic search
- …and more.

These tools simplify refining searches and pinpointing relevant information in real-time.

## Scripting
Scripting makes custom data manipulation and transformation possible during search and aggregation processes. Using scripting languages like Painless, you can calculate custom metrics, perform conditional logic, or adjust data dynamically in search time. This flexibility ensures tailored insights specific to your needs.

## Aggregations
Aggregations provide advanced data analysis, enabling you to extract actionable insights. With aggregations, you can calculate statistical metrics (for example, sums, averages, medians), group data into buckets (histograms, terms, and so on), or perform nested and multi-level analyses. Aggregations transform raw data into structured insights with ease.

## Geospatial Analysis
The geospatial capabilities enable analysis of location-based data, including distance calculations, polygon and bounding box queries, and geohash grid aggregations. This functionality is necessary for logistics, real estate, and IoT industries, where location matters.

## Machine Learning
Elasticsearch integrates machine learning for proactive analytics, helping you to:
- Detect anomalies in time-series data
- Forecast future trends
- Analyze seasonal patterns
- Perform powerful NLP operations such as semantic search
- Machine learning models simplify complex predictive tasks, unlocking new opportunities for optimization.

## Discover
Discover lets you interact directly with raw data. Use Discover to:
- Browse documents in your indices
- Apply filters and search queries
- Visualize results in real-time

It’s the starting point for exploratory analysis.

## Dashboards
Dashboards serve as centralized hubs for visualizing and monitoring data insights. With Dashboards, you can:
- Combine multiple visualizations into a single, unified view
- Display data from multiple indices or datasets for comprehensive analysis
- Customize layouts to suit specific workflows and preferences

Dashboards provide an interactive and cohesive environment to explore trends and metrics at a glance.

## Panels and visualizations
Panels and visualizations are the core elements that populate your dashboards, enabling dynamic data representation. They support diverse chart types, Interactive filtering, and drill-down capabilities to explore data further. These building blocks transform raw data into clear, actionable visuals, allowing users to analyze and interpret results effectively.

## Reporting and sharing
You can share your work and findings with colleagues and stakeholders or generate reports. Report generation can be scheduled or on-demand. You can choose from multiple formats (for example, PDF, CSV). These tools ensure that actionable insights reach the right people at the right time.
Alerting
You can set up alerts to monitor your data continuously. Alerts notify you when specific conditions are met. This ensures timely action on critical issues.

## Bringing it all together
Elasticsearch's features integrate seamlessly, offering an end-to-end solution for exploring, analyzing, and acting on data. If you want to explore any of the listed features in greater depth, refer to their respective documentation pages and check the provided hands-on examples and tutorials.

2 changes: 1 addition & 1 deletion explore-analyze/query-filter.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ Refer to [Query languages](#search-analyze-query-languages) for a full overview

[Query DSL](query-filter/languages/querydsl.md) is a full-featured JSON-style query language that enables complex searching, filtering, and aggregations. It is the original and most powerful query language for {{es}} today.

The [`_search` endpoint](../solutions/search/querying-for-search-searching-with-the-search-api.md) accepts queries written in Query DSL syntax.
The [`_search` endpoint](../solutions/search/querying-for-search.md) accepts queries written in Query DSL syntax.


#### Search and filter with Query DSL [search-analyze-query-dsl-search-filter]
Expand Down
44 changes: 42 additions & 2 deletions explore-analyze/query-filter/filtering.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ mapped_urls:
- https://www.elastic.co/guide/en/kibana/current/set-time-filter.html
---

# Filtering
# Filtering in Kibana

% What needs to be done: Write from scratch

Expand All @@ -15,4 +15,44 @@ mapped_urls:

% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc):

$$$_finding_your_apps_and_objects$$$
$$$_finding_your_apps_and_objects$$$

This page describes the common ways Kibana offers in most apps for filtering data and refining your initial search queries.

Some apps provide more options, such as [Dashboards](../dashboards.md).

## Time filter [set-time-filter]

Display data within a specified time range when your index contains time-based events, and a time-field is configured for the selected [{{data-source}}](../find-and-organize/data-views.md). The default time range is 15 minutes, but you can customize it in [Advanced Settings](https://www.elastic.co/guide/en/kibana/current/advanced-options.html).

1. Click ![calendar icon](../../images/kibana-time-filter-icon.png).
2. Choose one of the following:

* **Quick select**. Set a time based on the last or next number of seconds, minutes, hours, or other time unit.
* **Commonly used**. Select a time range from options such as **Last 15 minutes**, **Today**, and **Week to date**.
* **Recently used date ranges**. Use a previously selected data range.
* **Refresh every**. Specify an automatic refresh rate.

:::{image} ../../../images/kibana-time-filter.png
:alt: Time filter menu
:width: 300px
:::

3. To set start and end times, click the bar next to the time filter. In the popup, select **Absolute**, **Relative** or **Now**, then specify the required options.

:::{image} ../../../images/kibana-time-relative.png
:alt: Time filter showing relative time
:class: screenshot
:::

The global time filter limits the time range of data displayed. In most cases, the time filter applies to the time field in the data view, but some apps allow you to use a different time field.

Using the time filter, you can configure a refresh rate to periodically resubmit your searches.

To manually resubmit a search, click the **Refresh** button. This is useful when you use Kibana to view the underlying data.

## Additional filters [autocomplete-suggestions]

Structured filters are a more interactive way to create {{es}} queries, and are commonly used when building dashboards that are shared by multiple analysts. Each filter can be disabled, inverted, or pinned across all apps. Each of the structured filters is combined with AND logic on the rest of the query.

![Add filter popup](../../images/kibana-add-filter-popup.png "")
2 changes: 1 addition & 1 deletion explore-analyze/query-filter/languages.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ Refer to [Query languages](../query-filter.md#search-analyze-query-languages) fo

[Query DSL](languages/querydsl.md) is a full-featured JSON-style query language that enables complex searching, filtering, and aggregations. It is the original and most powerful query language for {{es}} today.

The [`_search` endpoint](../../solutions/search/querying-for-search-searching-with-the-search-api.md) accepts queries written in Query DSL syntax.
The [`_search` endpoint](../../solutions/search/querying-for-search.md) accepts queries written in Query DSL syntax.


#### Search and filter with Query DSL [search-analyze-query-dsl-search-filter]
Expand Down
10 changes: 8 additions & 2 deletions explore-analyze/query-filter/tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,18 @@ mapped_pages:

# Query tools [devtools-kibana]

**Dev Tools** contains tools that you can use to interact with your data.
Elasticsearch offers tools that you can use to query your data, manage those queries, and optimize them to be as efficient as possible.

| | |
| --- | --- |
| [Console](tools/console.md) | Interact with the REST APIs of {{es}} and {{kib}}, including sending requestsand viewing API documentation. |
| [Saved queries](tools/saved-queries.md) | Save your searches and queries to reuse them later. |
| [Console](tools/console.md) | Interact with the REST APIs of {{es}} and {{kib}}, including sending requests and viewing API documentation. |
| [{{searchprofiler}}](tools/search-profiler.md) | Inspect and analyze your search queries. |
| [Grok Debugger   ](tools/grok-debugger.md) | Build and debug grok patterns before you use them in your data processing pipelines. |
| [Painless Lab](../scripting/painless-lab.md) | [beta] Test and debug Painless scripts in real-time. |
| [Playground](tools/playground.md) | Combine your Elasticsearch data with the power of large language models (LLMs) for retrieval augmented generation (RAG), using a chat interface. |





2 changes: 1 addition & 1 deletion explore-analyze/query-filter/tools/playground.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,6 @@ Use the Search Playground to test and edit {{es}} queries visually in the UI. Th
Find Playground in the {{es-serverless}} UI under **{{es}} > Build > Playground**.

::::{note}
ℹ️ The Playground documentation currently lives in the [{{kib}} docs](../../../solutions/search/rag/playground.md).
ℹ️ For more details, check the full [Playground documentation](../../../solutions/search/rag/playground.md).

::::
Loading

0 comments on commit cd5369e

Please # to comment.