PerfOps Alerts: what’s so great and how to integrate (Alerts usage, Slack and Opsgenie notifications setup)
As you may know, one of the major PerfOps activities is the statistics collection. This is implemented with more than 300 nodes that perform synthetic tests of DNS and Cloud Providers performance every minute and the Real User Monitoring, that tests CDN Providers and recently went over 1 billion tests per day!
The PerfOps Alerts is a special notification system that is set on top of collected RUM and Synthetic data and notifies users in case of provider availability or performance degradation.
Alerts will come in handy in various situations:
- you are a DNS or CDN provider owner and you want to monitor your service performance and uptime;
- you are building/going to build own complex multi-CDN or multi-Cloud location based infrastructure and need some initial statistics to configure it the best possible way;
- you are testing your DNS Resolver and need to monitor its quality from particular ASN;
- you are just curious :)
Let’s start with the first question: “what’s so great” and take a short Alerts tour.
Setting up Alert: providers, conditions, location- ASN-ISP filters
Couple of words of “how it works”. You create new Alert, select CDN (or DNS or Cloud) Provider, set up the rules, for example ‘trigger if Performance (Latency) goes equal or above 40ms’, when it is triggered — you get a notification, when it goes back to normal (below 40ms) — you also get a notification.
Our example case will be: ‘Monitor Cloudflare DNS Provider statistics from Europe, with 1 minute interval, the Performance threshold should be greater or equal 8ms’.
Which, in simple words, means: if per-minute Perfops statistics for Cloudflare DNS Provider Performance is equal or goes above 8 milliseconds — we receive notification. When it goes back below 8 ms — we receive ‘back to normal’ notification.
Let’s create a new Alert. Go to the Alerts page, click ‘Create new alert’ button and get the Alert pop-up window.
As you can see- it is quite clear and intuitive — select DNS Providers from Data sources, the Provider from the list, last one minute interval, Performance and condition. Nothing complicated.
At the Location section — select Europe as Continent (note, that multiple continents can be selected):
We should pay more attention to that Location section. If You used PerfOps Alerts before — you remember that only Continent and Country filters were available. Today, the situation has changed, the ability to filter by City, ASN and ISP has been added:
So, now there are not only Continent and Country, but three more filters. You can choose the particular city from the list.
Notice, that for DNS/Cloud Providers only cities PerfOps has Synthetic nodes placed in are shown. For CDNs the list is much bigger because the Real User Monitoring that collects CDN Providers performance and uptime data worldwide is used:
Simply put, if you are monitoring CDN Providers performance or uptime — you are able to select almost every city on the planet, isn’t it great?
And that is not all: now you can add ISP and ASN filters so your monitoring becomes as accurate as possible.
ISP filter:
and ASN filter:
The lists of ASNs and ISPs are also based on real metrics received from PerfOps test nodes (DNS and Cloud providers performance) and via Real User Monitoring (CDN Providers statistics).
A short video that illustrates the usage of that new Alerts features was recorded, you definitely want to watch it :)
And, of course, you can preview your results, it is very helpful when you determine the threshold number (‘greater’, ‘greater or equal’ etc.) :
Ok, everything is set up, you fill in email for notifications and save your alert:
So, now all notifications when your Alert is triggered and when it goes back to normal will be sent to that email. And you can see, that email is not the only option here.
The PerfOps team has also added Alerts integration for such well-known business communication platform as Slack and powerful incident management platform as Opsgenie.
PerfOps Alerts Integration with Slack and Opsgenie
The integration process is more than simple. For Slack you should have a link to your Slack incoming webhook URL. Simply paste it to the corresponding field:
After saving it shows Slack as the alert notification channel:
The Alert is saved and you start receiving Alerts notifications. If the event was triggered, you receive a ‘red’ notification:
It means that the DNS provider performance went above 8ms. The message also shows all Alert information (condition, trigger value, interval, locations).
And when it goes back to ‘normal’ (in our case latency goes below 8ms), the ‘green’ notification comes up:
Now, let’s set the same Alert we modified before to Opsgenie channel. Simply edit the alert and change its Name (To ‘Test Alert Opsgenie’, just to avoid any confusions) and Notification channel info.
Fill in your Opsgenie API URL and Key:
Save the Alert, you can see that its Notification channel is changed:
And you start receiving notifications (when DNS provider performance goes above 8ms):
In details:
So, that’s how it works!
Note: being part of the Enterprise plan, City, ASN and ISP Alerts filters are freely available for PerfOps users until February, 2021, if you are interested in those, feel free to book a call with PerfOps sales team.
