Create alerts with OpenSearch Dashboards¶
OpenSearch alerting feature sends notifications when data from one or more indices meets certain conditions that can be customized. Use case examples are such as monitoring for HTTP status code 503, CPU load average above certain percentage or watch for counts of a specific keyword in logs for a specific amount of interval, notification to be configured to be sent via email, slack or custom webhooks and other destination.
In the following example we are using Slack as the destination and a
sample-host-health index as datasource to create a simple alert to check cpu load. An action will be triggered when average of
3 minutes is above
Create using Dashboards UI¶
In order to create an alert via OpenSearch Dashboards interface, follow these steps:
Log in to the Aiven web console and select your OpenSearch service.
Click the Overview tab and under Connection Information click on OpenSearch Dashboards tab.
This opens the OpenSearch Dashboard
In OpenSearch Dashboard open the left side panel and under OpenSearch Plugins click on Alerting.
To configure each alert the following needs to be created, we will walk-through configuration of each section.
Create a destination¶
Destination is a location for notifications to be delivered when an action is triggered.
Open the Destination tab and click on Add destination
Fill in the fields under Destination
slack-testas the Name
Paste your slack webhook here
https://your_slack_webhook_URLunder Webhook URL
Destination Type can be:
Custom webhook or
When using email you need to have a SMTP server configured for a valid domain to deliver email notifications
Create a monitor¶
Monitor is a job that runs on a defined schedule and queries OpenSearch indices.
Open the Monitors tab and click on Create monitor
Fill in the fields under Monitor details
High CPU Monitorinto Monitor name
Per query monitoras the Monitor type
Visual editoras the Monitor defining method
By intervalunder Schedule Frequency
Under Run every select
Schedule Frequency can be By internal, Daily, Weekly, Monthly, Custom CRON expression
Fill in the fields under Data source
Data source is the OpenSearch indices to query.
timestampinto Time field
Query defines the fields to query from indices and how to evaluate the results.
Under Metrics click on Add metric
average()under Aggregation and
cpu_usage_percentageunder Field, click on Save
3under Time range for the last and select
Create a trigger¶
Triggers is a defined conditions from the queries results from monitor. If conditions are met, alerts are generated.
Click on Add trigger
high_cpuas the Trigger name
1 (Highest)for Severity level
Under Trigger condition select
IS ABOVEfrom the drop-down menu and fill
75into the number field
You can see a visual graph below trigger with the index data and the trigger condition you have defined as a red line
Fill in the fields under Actions
Actions defines the destination for notification alerts when trigger conditions are met.
slackas Action name
High CPU Test Alertas Message subject
Multiple Actions can be defined, in this example we will define one action to send notification to destination we have defined in step 4
Message can be adjusted as needed, check Message Preview to see the sample and use Send test message to validate notification delivery
Click on Create and your monitor is ready!
For further details on alerting monitors configuration