Month: April 2016

To monitor my Linux servers, I have been using icinga since its forking from the popular Nagios. Icinga has really matured over the last few years and currently at version 2.4.4, I would recommend anyone serious at network and server monitoring to check this open source project. Installation and setup of icinga2 is somewhat complex, but if you are an experienced command line sort of guy, its not that hard at all. In fact the introduction of icingaweb2 has made it possible to have an easy to use interface to setup your monitoring needs. I will not delve into the nitty gritty of install and setup. For that please take a look at the detailed documentation.

Several alerts are possible when using Icinga including e-mail, sms, XMPP, IRC, twitter etc. I wanted a way to get notified of host and service failures using telegram. Luckily, telegram provides an API to do just that.

The secret source in all of this; is to install a command line interface for telegram called telegram-cli . In order for telegram-cli to compile without any issues, ensure that all dependencies are installed

For Fedora:

sudo dnf -y install
lua-devel openssl-devel libconfig-devel readline-devel libevent-devel
jansson-devel python-devel

For Ubuntu/Debian:

sudo apt-get install
libreadline-dev libconfig-dev libssl-dev lua5.2 liblua5.2-dev
libevent-dev libjansson-dev libpython-dev make 

Now install telegram-cli

cd /usr/src && git clone –recursive https://github.com/vysheng/tg.git

Note: Make sure you add the recursive flag to git clone, otherwise make will fail

cd tg

./configure && make

The binary is under ./bin/telegram-cli. Simply create a symbolic link to it.

cd /usr/sbin && ln -s /usr/src/tg/bin/telegram-cli .

Thats it for the installation of telegram-cli. Please read the README file in the sources root folder, especially on how to use this application.

Icinga2 telegram-cli setup

First ensure that the user icinga (created during icinga install process) can send a message using telegram-cli. Remember all alerts go out under the icinga user, so you have to get this part working first, lest you spend hours troubleshooting.

Temporarily change the shell for icinga from ‘/sbin/nologin’ to ‘/bin/bash.’ Remember to change back to nologin for icinga user when testing is completed.

usermod -s /bin/bash icinga

Now change to user icinga: Assumming that you are root user, if not you will be prompted for a password, which you would setup as root.

su -l icinga

telegram-cli

Enter your phone number and SMS code sent when prompted

Then test to see if you can send a message:

telegram-cli -D -W -e "msg [PEER_NAME] 'Hello World'"

If it works, then you can proceed to configure icinga2.

Create notification script for services

cat > /etc/icinga2/scripts/notify_services_via_telegram.sh << EOF

#!/usr/bin/env bash

TELEGRAM_CMD=/usr/sbin/telegram-cli

TELEGRAM_OPTS=”-D -W”

TELEGRAM_PEER=[PEER_NAME]

TELEGRAM_MSG=$(cat <<TEMPLATE

[$HOSTALIAS] $NOTIFICATIONTYPE: $SERVICEDESC is $SERVICESTATE

TEMPLATE

)

$TELEGRAM_CMD $TELEGRAM_OPTS -e “msg $TELEGRAM_PEER ‘$TELEGRAM_MSG'”

EOF

Ensure that above script is executable:

chmod 755 /etc/icinga2/scripts/notify_services_via_telegram.sh

Proceed to configure commands, notifications, and hosts config files: Note: Am only setting up notifications for services. You can do the same for hosts information.

cat >> /etc/icinga2/conf.d/commands.conf <<EOF

/** Telegram */

object NotificationCommand “telegram-service-notification” {

import “plugin-notification-command”

command = [ SysconfDir + “/icinga2/scripts/notify_services_via_telegram.sh” ]

env = {

“NOTIFICATIONTYPE” = “$notification.type$”

“SERVICEDESC” = “$service.name$”

“HOSTALIAS” = “$host.display_name$”,

“HOSTADDRESS” = “$address$”,

“SERVICESTATE” = “$service.state$”,

“LONGDATETIME” = “$icinga.long_date_time$”,

“SERVICEOUTPUT” = “$service.output$”,

“NOTIFICATIONAUTHORNAME” = “$notification.author$”,

“NOTIFICATIONCOMMENT” = “$notification.comment$”,

“HOSTDISPLAYNAME” = “$host.display_name$”,

“SERVICEDISPLAYNAME” = “$service.display_name$”,

“USEREMAIL” = “$user.email$”

}

}

EOF

cat >> /etc/icinga2/conf.d/notifications.conf <<EOF

/** Telegram */

apply Notification “telegram-icingaadmin” to Service {

import “telegram-service-notification”

user_groups = host.vars.notification.telegram.groups

users = host.vars.notification.telegram.users

assign where host.vars.notification.telegram

}

EOF

To get alerts on services for a host, add the following stanza to the host configuration in /etc/icinga2/conf.d/hosts.conf

/* Telegram */

vars.notification[“telegram”] = {

groups = [ “icingaadmins” ]

}

Finally, check your config files

icinga2 daemon -C

Restart icinga if all is OK. You will now be notified via telegram when services fail for the above configured host.