How to Set Up Nagios Alerts on a Dedicated Server?
Nagios is a powerful monitoring system that allows you to track the status of servers, services, and network equipment. However, to make Nagios truly useful, you need to properly configure the alert system. This article provides a detailed guide on configuring Nagios alerts on a dedicated server so you can quickly respond to any issues and minimize downtime. We will explore various ways to configure alerts, including using email and SMS, as well as integration with other systems.
Contents:
- Configuring Email Alerts
- Configuring SMS Alerts
- Integration with Slack and Telegram
- Configuring Alert Escalation
Configuring Email Alerts
Configuring email alerts is one of the most common and straightforward ways to receive notifications from Nagios. This method allows you to receive detailed reports on the status of your servers and services directly in your email. In this section, we will thoroughly examine how to configure sending email alerts from Nagios, what parameters to consider, and what common issues may arise.
Configuring Email Parameters in Nagios
The first step is configuring the email parameters in the Nagios configuration files. The main parameters that need to be specified include the SMTP server, port, credentials (if required), and sender address. These parameters are usually specified in the /etc/nagios/nagios.cfg file. Open this file with a text editor and find the section dedicated to email configuration.
# grep email /etc/nagios/nagios.cfg
service_notification_options w,u,c,r,f,s
host_notification_options d,u,r,f,s
email_notification_options w,u,c,r,f,s
Ensure that the parameters listed above are enabled. They determine when notifications are sent (e.g., on warning (w), critical error (c), recovery (r)).
Next, you need to configure the command to send the email. Typically, the notify-service-by-email and notify-host-by-email commands, defined in the /etc/nagios/commands.cfg file, are used.
# grep notify-service-by-email /etc/nagios/commands.cfg
define command {
command_name notify-service-by-email
command_line /usr/bin/printf "%b" "*** Nagios *\n\nNotification Type: $NOTIFICATIONTYPE$\n\nService: $SERVICEDESC$\nHost: $HOSTNAME$\nAddress: $HOSTADDRESS$\nState: $SERVICESTATE$\n\nDate/Time: $LONGDATETIME$\n\nAdditional Info:\n\n$SERVICEOUTPUT$\n" | /usr/bin/mail -s " $NOTIFICATIONTYPE$ Service Alert: $HOSTNAME$/$SERVICEDESC$ is $SERVICESTATE$ " $CONTACTEMAIL$
}
# grep notify-host-by-email /etc/nagios/commands.cfg
define command {
command_name notify-host-by-email
command_line /usr/bin/printf "%b" "* Nagios *\n\nNotification Type: $NOTIFICATIONTYPE$\n\nHost: $HOSTNAME$\nState: $HOSTSTATE$\nAddress: $HOSTADDRESS$\n\nDate/Time: $LONGDATETIME$\n\nAdditional Info:\n\n$HOSTOUTPUT$\n" | /usr/bin/mail -s " $NOTIFICATIONTYPE$ Host Alert: $HOSTNAME$ is $HOSTSTATE$ **" $CONTACTEMAIL$
}
Important: Make sure you have the mail package or similar installed, which is necessary for sending mail from the command line. For example, in Debian/Ubuntu: sudo apt-get install mailutils. In CentOS/RHEL: sudo yum install mailx.
Configuring Contacts to Receive Alerts
After configuring the email parameters, you need to define the contacts that will receive alerts. This is done in the /etc/nagios/conf.d/contacts.cfg file (or in a separate file specified in nagios.cfg). You need to create a contact definition, specifying their name, email address, and other parameters.
define contact {
contact_name nagiosadmin
alias Nagios Admin
email nagios@example.com
service_notification_period 24x7
host_notification_period 24x7
service_notification_options w,u,c,r,f,s
host_notification_options d,u,r,f,s
service_notification_commands notify-service-by-email
host_notification_commands notify-host-by-email
}
In this example, we created a contact named nagiosadmin, specified the email address nagios@example.com, defined the notification period (24×7), and specified the commands that will be used to send notifications by email.
Expert Tip: To prevent your mailbox from being flooded, consider using filters in your email program to automatically sort Nagios notifications.
Linking Contacts to Hosts and Services
The final step is linking the created contacts to the hosts and services for which you want to receive alerts. This is done in the host and service definitions in the /etc/nagios/conf.d/hosts.cfg and /etc/nagios/conf.d/services.cfg files (or in the corresponding files specified in nagios.cfg).
define host {
use linux-server
host_name webserver1
alias Web Server 1
address 192.168.1.100
contacts nagiosadmin
}
define service {
use generic-service
host_name webserver1
service_description HTTP
check_command check_http
contacts nagiosadmin
}
In these examples, we linked the contact nagiosadmin to the host webserver1 and the service HTTP. Now, when the state of webserver1 or HTTP changes, nagiosadmin will receive an email notification.
After making changes to the Nagios configuration files, you need to restart the Nagios service for the changes to take effect:
sudo systemctl restart nagios
Example: Suppose you want to configure alerts for disk space on the fileserver server. You need to add a service to the /etc/nagios/conf.d/services.cfg file:
define service {
use generic-service
host_name fileserver
service_description Disk Space
check_command check_disk!20%!10%
contacts nagiosadmin
}
In this example, we use the check_disk command to monitor disk space. The parameters 20%!10% indicate the warning and critical error levels (20% and 10% free space, respectively). After adding this service and restarting Nagios, you will receive email notifications when the disk space on fileserver approaches these levels.
Important: Always check the syntax of the Nagios configuration files before restarting the service. You can do this using the command nagios -v /etc/nagios/nagios.cfg. This command will check the configuration files for errors and output the corresponding messages.
Configuring SMS Alerts
SMS alerts are an effective way to receive notifications about critical issues, especially when you don’t have access to email. Configuring SMS alerts requires integration with an external SMS gateway service. In this section, we will explore various ways to configure SMS alerts, including using free and paid services, as well as configuring commands and contacts in Nagios.
Choosing an SMS Gateway
The first step is to choose a suitable SMS gateway. There are many services that offer SMS gateway services, both paid and free. Paid services usually offer higher reliability and message delivery speed, as well as broader customization options. Free services may have limitations on the number of messages sent or delays in delivery.
Some popular SMS gateways:
- Twilio
- Nexmo (Vonage)
- Clickatell
- TextMagic
For testing purposes, you can use free SMS gateways, but for production systems, it is recommended to use paid services.
Configuring a Command to Send SMS
After selecting an SMS gateway, you need to configure a command in Nagios to send SMS messages. This command will use the API of the selected SMS gateway to send messages. You need to create a new command in the /etc/nagios/commands.cfg file.
Example: Configuring a command to send SMS via Twilio:
define command {
command_name notify-service-by-sms
command_line /usr/bin/curl -X POST 'https://api.twilio.com/2010-04-01/Accounts/ACxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/Messages.json' \
--data-urlencode 'To=$CONTACTPAGER$' \
--data-urlencode 'From=+1234567890' \
--data-urlencode 'Body=Nagios: $NOTIFICATIONTYPE$ - $HOSTNAME$ - $SERVICEDESC$ is $SERVICESTATE$' \
-u ACxxxxxxxxxxxxxxxxxxxxxxxxxxxxx:your_auth_token
}
define command {
command_name notify-host-by-sms
command_line /usr/bin/curl -X POST 'https://api.twilio.com/2010-04-01/Accounts/ACxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/Messages.json' \
--data-urlencode 'To=$CONTACTPAGER$' \
--data-urlencode 'From=+1234567890' \
--data-urlencode 'Body=Nagios: $NOTIFICATIONTYPE$ - $HOSTNAME$ is $HOSTSTATE$' \
-u ACxxxxxxxxxxxxxxxxxxxxxxxxxxxxx:your_auth_token
}
In this example, replace ACxxxxxxxxxxxxxxxxxxxxxxxxxxxxx with your Account SID from Twilio, +1234567890 with your Twilio phone number, and your_auth_token with your Auth Token from Twilio. Also, make sure you have the curl package installed.
Important: Store your API credentials in a safe place and do not publish them publicly.
Configuring Contacts to Receive SMS Alerts
Now you need to configure the contacts that will receive SMS alerts. This is done in the /etc/nagios/conf.d/contacts.cfg file. You need to specify the contact’s phone number in the pager parameter.
define contact {
contact_name admin_sms
alias Admin SMS
pager +79991234567
service_notification_period 24x7
host_notification_period 24x7
service_notification_options w,u,c,r,f,s
host_notification_options d,u,r,f,s
service_notification_commands notify-service-by-sms
host_notification_commands notify-host-by-sms
}
In this example, we created a contact named admin_sms and specified the phone number +79991234567. Note that the phone number must be specified in international format.
Example: If you are using Nexmo, the command to send SMS will look like this:
define command {
command_name notify-service-by-sms
command_line /usr/bin/curl -d "api_key=YOUR_API_KEY&api_secret=YOUR_API_SECRET&to=$CONTACTPAGER$&from=Nagios&text=Nagios: $NOTIFICATIONTYPE$ - $HOSTNAME$ - $SERVICEDESC$ is $SERVICESTATE$" 'https://rest.nexmo.com/sms/json'
}
define command {
command_name notify-host-by-sms
command_line /usr/bin/curl -d "api_key=YOUR_API_KEY&api_secret=YOUR_API_SECRET&to=$CONTACTPAGER$&from=Nagios&text=Nagios: $NOTIFICATIONTYPE$ - $HOSTNAME$ is $HOSTSTATE$" 'https://rest.nexmo.com/sms/json'
}
Replace YOUR_API_KEY and YOUR_API_SECRET with your credentials from Nexmo.
Expert Tip: Use the SMS sending limits provided by your SMS gateway to avoid unexpected costs.
Linking Contacts to Hosts and Services
As with email alerts, you need to link contacts to the hosts and services for which you want to receive SMS alerts. This is done in the host and service definitions in the /etc/nagios/conf.d/hosts.cfg and /etc/nagios/conf.d/services.cfg files.
define host {
use linux-server
host_name dbserver
alias Database Server
address 192.168.1.101
contacts admin_sms
}
define service {
use generic-service
host_name dbserver
service_description Database Connection
check_command check_tcp!3306
contacts admin_sms
}
After making changes to the Nagios configuration files, you need to restart the Nagios service:
sudo systemctl restart nagios
Important: Test sending SMS alerts to make sure they are working correctly. You can manually trigger an error to generate an alert.
Integration with Slack and Telegram
Integrating Nagios with messaging platforms like Slack and Telegram allows you to receive real-time notifications of issues directly in your communication channels. This can significantly speed up the incident response process. In this section, we will explore how to configure Nagios integration with Slack and Telegram to receive notifications about the status of your servers and services.
Integration with Slack
To integrate Nagios with Slack, you need to create a Slack App and configure an incoming webhook. A webhook is a URL to which Nagios will send notifications in JSON format.
Steps to configure integration with Slack:
- Create a Slack App in your Slack workspace.
- Activate Incoming Webhooks in your Slack App settings.
- Get the webhook URL.
- Configure a command in Nagios to send notifications to the webhook.
Example: Configuring a command in /etc/nagios/commands.cfg to send notifications to Slack:
define command {
command_name notify-service-by-slack
command_line /usr/bin/curl -X POST --data-urlencode "payload={\"channel\": \"#nagios-alerts\", \"username\": \"Nagios\", \"text\": \"*$NOTIFICATIONTYPE* Service Alert: $HOSTNAME$/$SERVICEDESC$ is $SERVICESTATE$ - $SERVICEOUTPUT$\", \"icon_emoji\": \":warning:\"}" YOUR_SLACK_WEBHOOK_URL
}
define command {
command_name notify-host-by-slack
command_line /usr/bin/curl -X POST --data-urlencode "payload={\"channel\": \"#nagios-alerts\", \"username\": \"Nagios\", \"text\": \"*$NOTIFICATIONTYPE* Host Alert: $HOSTNAME$ is $HOSTSTATE$ - $HOSTOUTPUT$\", \"icon_emoji\": \":warning:\"}" YOUR_SLACK_WEBHOOK_URL
}
Replace YOUR_SLACK_WEBHOOK_URL with the webhook URL you obtained when configuring the Slack App. The channel parameter specifies the channel to which notifications will be sent. The icon_emoji parameter specifies the emoji that will be displayed next to the notification.
Next, configure contacts in /etc/nagios/conf.d/contacts.cfg, specifying the commands for sending notifications to Slack:
define contact {
contact_name admin_slack
alias Admin Slack
email ignore@example.com
service_notification_period 24x7
host_notification_period 24x7
service_notification_options w,u,c,r,f,s
host_notification_options d,u,r,f,s
service_notification_commands notify-service-by-slack
host_notification_commands notify-host-by-slack
}
Finally, link the contact admin_slack to the hosts and services for which you want to receive notifications in Slack.
Important: Slack requires that the webhook URL be protected by the HTTPS protocol. Make sure your server is configured to use HTTPS.
Integration with Telegram
To integrate Nagios with Telegram, you need to create a Telegram Bot and get its API Token and Chat ID. The API Token is a unique identifier for your bot, and the Chat ID is the identifier of the chat to which notifications will be sent.
Steps to configure integration with Telegram:
- Create a Telegram Bot using BotFather in Telegram.
- Get the API Token of your bot.
- Get the Chat ID of the chat to which you want to send notifications.
- Configure a command in Nagios to send notifications via the Telegram API.
Example: Configuring a command in /etc/nagios/commands.cfg to send notifications to Telegram:
define command {
command_name notify-service-by-telegram
command_line /usr/bin/curl -s -X POST "https://api.telegram.org/botYOUR_TELEGRAM_BOT_TOKEN/sendMessage" -d "chat_id=YOUR_TELEGRAM_CHAT_ID&text=*$NOTIFICATIONTYPE*%20Service%20Alert:%20$HOSTNAME$/$SERVICEDESC$%20is%20$SERVICESTATE$%20-%20$SERVICEOUTPUT$"
}
define command {
command_name notify-host-by-telegram
command_line /usr/bin/curl -s -X POST "https://api.telegram.org/botYOUR_TELEGRAM_BOT_TOKEN/sendMessage" -d "chat_id=YOUR_TELEGRAM_CHAT_ID&text=*$NOTIFICATIONTYPE*%20Host%20Alert:%20$HOSTNAME$%20is%20$HOSTSTATE$%20-%20$HOSTOUTPUT$"
}
Replace YOUR_TELEGRAM_BOT_TOKEN with the API Token of your bot, and YOUR_TELEGRAM_CHAT_ID with the Chat ID of the chat. The %20 characters are used to encode spaces in the URL.
Next, configure contacts in /etc/nagios/conf.d/contacts.cfg, specifying the commands for sending notifications to Telegram:
define contact {
contact_name admin_telegram
alias Admin Telegram
email ignore@example.com
service_notification_period 24x7
host_notification_period 24x7
service_notification_options w,u,c,r,f,s
host_notification_options d,u,r,f,s
service_notification_commands notify-service-by-telegram
host_notification_commands notify-host-by-telegram
}
Finally, link the contact admin_telegram to the hosts and services for which you want to receive notifications in Telegram.
Example: If you want to send notifications to Telegram in Markdown format, use the parse_mode=Markdown parameter in the request:
define command {
command_name notify-service-by-telegram
command_line /usr/bin/curl -s -X POST "https://api.telegram.org/botYOUR_TELEGRAM_BOT_TOKEN/sendMessage" -d "chat_id=YOUR_TELEGRAM_CHAT_ID&text=*$NOTIFICATIONTYPE*%20Service%20Alert:%20$HOSTNAME$/$SERVICEDESC$%20is%20$SERVICESTATE$%20-%20$SERVICEOUTPUT$&parse_mode=Markdown"
}
Expert Tip: Use different channels or chats for different types of notifications to make filtering and prioritization easier.
Comparison of notification methods:
| Notification Method | Advantages | Disadvantages |
|---|---|---|
| Detailed information, easy to set up | Delivery delay, may be missed | |
| SMS | Fast delivery, reliability | Limited message size, cost |
| Slack/Telegram | Instant notifications, collaboration | Requires internet connection, can be distracting |
Configuring Alert Escalation
Alert escalation is a mechanism that allows sending notifications to different groups of people depending on the severity of the problem and the time elapsed since its detection. This ensures that critical issues are not ignored and are resolved as quickly as possible. In this section, we will explore how to configure alert escalation in Nagios so that notifications are sent to different contacts or contact groups depending on various conditions.
Defining Notification Timeperiods
The first step in configuring alert escalation is to define notification timeperiods. Timeperiods define what time of day and which days of the week notifications should be sent. This avoids sending notifications during non-working hours if it is not necessary.
Timeperiods are defined in the /etc/nagios/conf.d/timeperiods.cfg file (or in a separate file specified in nagios.cfg).
define timeperiod {
timeperiod_name 24x7
alias 24 Hours A Day, 7 Days A Week
use generic-timeperiod
start_time 00:00
end_time 24:00
}
define timeperiod {
timeperiod_name workhours
alias Work Hours
use generic-timeperiod
monday 09:00-18:00
tuesday 09:00-18:00
wednesday 09:00-18:00
thursday 09:00-18:00
friday 09:00-18:00
}
In this example, we defined two notification timeperiods: 24x7 (around the clock) and workhours (working hours from Monday to Friday).
Defining Contact Groups
The next step is to define contact groups. Contact groups allow you to combine multiple contacts into one group to simplify notification management. This is useful when you need to send notifications to multiple people simultaneously.
Contact groups are defined in the /etc/nagios/conf.d/contacts.cfg file.
define contactgroup {
contactgroup_name admins
alias Nagios Administrators
members nagiosadmin, admin_sms, admin_telegram
}
define contactgroup {
contactgroup_name developers
alias Developers Team
members developer1, developer2
}
In this example, we defined two contact groups: admins (administrators) and developers (developers). The admins group includes the contacts nagiosadmin, admin_sms, and admin_telegram, and the developers group includes the contacts developer1 and developer2.
Configuring Escalation for Hosts and Services
Now you need to configure escalation for hosts and services. This is done in the host and service definitions in the /etc/nagios/conf.d/hosts.cfg and /etc/nagios/conf.d/services.cfg files. The notification_interval, first_notification_delay, notification_period, and contacts/contact_groups parameters are used to configure escalation.
Example: Configuring escalation for a host:
define host {
use linux-server
host_name important_server
alias Important
