Creating Alert Notifications in Elasticsearch: A Comprehensive Guide to Slack Integration

Haydar Külekci
4 min readJul 4, 2023

I used many different tools to create an alert service for the logs. In Elasticsearch, you can easily create alerts and even send them to Slack. In this article, I will give you step by step explanation for creating an alert to send messages to the Slack channel. So, before creating alerts, we assume that we have already connected our APM or Fleet services or a filebeat connected to the Elasticsearch cluster. And we have a stream for the logs. In my case, I used filebeat directly to ingest nginx logs into our cluster. So, as a result, I have a data stream as filebeat-8.4.3 and there are the nginx logs inside the data stream. You can use the same way to create an alert for other indices. I will use Kibana UI to create alerts.

So, let’s begin …

Kibana has a couple of menus to start creating alerts. In this article, we will focus Observability> Alerts menu. This page will list the already created Alerts if you have already. So, on this page, you can use the “Manage Rules“ page to create a rule for Alerts.

Observability > Alerts Page

After jumping into the Manage Rules page, you can create a rule by using Create Rule button on the right top.

Manage Rules Page

So, a popup will appear, and you need to provide a name and select a rule type. So, I picked up “Log threshold” as the rule type.

Or, you can use Observability > Logs > Stream page to create an alert rule as follow:

Observability > Logs > Stream Page

In both menus, you will see the same popup on the page. Just only whenever you use the Stream menu, “Log threshold” will be selected by default and unchangeable.

So, after providing the name and selecting the rule type, you need to have the following screen:

As you can see, according to the conditions, the rule looks for the documents in which their log.level field should be error. Conversely, the rule-looking range is for the last 5 minutes. The rule will create an alert if it counts more than five documents in this condition.

So, before saving the rule, let’s configure the Action part, too. In this example, as I mentioned above, we will create a connection to Slack. But as you can see below, there are so many other options:

List of Actions

When you select the Slack first time, you need to configure the Slack integration. And you will see the following popup:

There are two different integration types for the Slack connector. I used the webhook. Just open the Slack apps page and create an app for the integration. Gave an App Name:

Then later, activate the Incoming Webhooks and add a new webhook to workspace:

You need to select a channel. And the channel should be created already:

Then now, you have an integration Webhook URL.

Let’s copy it into our Slack Connector form and give the name “Elasticsearch Alert Channel” to our connector. After saving the connector, you will get some configuration about the connector.

So you can configure the frequency of the action. In this example, we will fire a message whenever the status changes. This status means we have five documents with an error field for the last 5 minutes, and we did not have more than 5 logs on our last look. In the recent look, we will fire an alert. So, now, we can save the Alert Rule, and you will see it in the list of Manage Rules page.

So, whenever you will get more than five errors on Nginx logs, you will get the following messages on Slack:

Lastly, you can follow how many alerts are generated through the time in the Observation > Alerts menu.

Thanks for reading.

If you find it useful, please don’t hesitate to applaud.

And also, follow me on Twitter and add me on Linkedin to reach more content.

--

--

Haydar Külekci

Elastic Certified Engineer - Open to new opportunities & seeking sponsorship for UK/Netherland relocation 🇳🇱🇬🇧 https://www.linkedin.com/in/hkulekci/