Icinga 2: Send SMS alerts only for CRITICAL status and its recoveries

Written by - 0 comments

Published on - Listed in Monitoring Nagios Icinga


Needed to find a way to configure Icinga 2, that only alerts with a CRITICAL state are sent by SMS (no WARNING or UNKNOWN states). However a recovery (OK) from a CRITICAL state should of course also be sent.

To handle this, I used the service.last_state and service.last_state_id macros, which contain exactly this information; what was the prior status of this service?

A special NotificationCommand definition for sending SMS alerts was added and it contains the last_state macros:

object NotificationCommand "notify-service-by-sms" {
        import "plugin-notification-command"
        command = [ SysconfDir + "/icinga2/scripts/sms-service-notification.sh" ]

        env = {
        NOTIFICATIONTYPE = "$notification.type$"
        SERVICEDESC = "$service.name$"
        HOSTALIAS = "$host.display_name$"
        HOSTADDRESS = "$address$"
        SERVICESTATE = "$service.state$"
        SERVICEPREVIOUSSTATE = "$service.last_state$"
        SERVICESTATEID = "$service.state_id$"
        SERVICEPREVIOUSSTATEID = "$service.last_state_id$"
        LONGDATETIME = "$icinga.long_date_time$"
        SHORTDATETIME = "$icinga.short_date_time$"
        SERVICEOUTPUT = "$service.output$"
        NOTIFICATIONAUTHORNAME = "$notification.author$"
        NOTIFICATIONCOMMENT = "$notification.comment$"
        HOSTDISPLAYNAME = "$host.display_name$"
        SERVICEDISPLAYNAME = "$service.display_name$"
        USEREMAIL = "$user.email$"
        USERPAGER = "$user.pager$"
        }
}

As you see, this NotificationCommand "notify-service-by-sms" will launch the script /etc/icinga2/scripts/sms-service-notification.sh and will send the enviornment variables mentioned to the script.

The script itself contains the logic when and how to send the SMS:

#!/bin/bash

# 20151109 Claudio: Only send SMS when $USERPAGER is set and when $SERVICESTATE is CRITICAL
# 20151110 Claudio: Also send OK (Recovery) when previous state was CRITICAL

sendsms() {
templatesms=`cat <<TEMPLATE
$NOTIFICATIONTYPE
Service: $SERVICEDESC on Host $HOSTALIAS is $SERVICESTATE
$SHORTDATETIME
TEMPLATE
`
/usr/bin/printf "%b" "$templatesms" | mailx $USERPAGER -- -f icinga@example.com
}

if ! [[ $USERPAGER = "" ]] ; then

  if [[ $SERVICESTATEID -eq 2 ]]; then sendsms
  elif [[ $SERVICESTATEID -eq 0 ]] && [[ $SERVICEPREVIOUSSTATEID -eq 2 ]]; then sendsms
  fi

fi

# Always send e-mail:

templatemail=`cat <<TEMPLATE
***** Icinga  *****

Notification Type: $NOTIFICATIONTYPE

Service: $SERVICEDESC
Host: $HOSTALIAS
Address: $HOSTADDRESS
State: $SERVICESTATE

Date/Time: $LONGDATETIME

Additional Info: $SERVICEOUTPUT

Comment: [$NOTIFICATIONAUTHORNAME] $NOTIFICATIONCOMMENT
TEMPLATE
`
/usr/bin/printf "%b" "$templatemail" | mailx -s "$NOTIFICATIONTYPE - $HOSTDISPLAYNAME - $SERVICEDISPLAYNAME is $SERVICESTATE" $USEREMAIL


Add a comment

Show form to leave a comment

Comments (newest first)

No comments yet.

RSS feed

Blog Tags:

  AWS   Android   Ansible   Apache   Apple   Atlassian   BSD   Backup   Bash   Bluecoat   CMS   Chef   Cloud   Coding   Consul   Containers   CouchDB   DB   DNS   Database   Databases   Docker   ELK   Elasticsearch   Filebeat   FreeBSD   Galera   Git   GlusterFS   Grafana   Graphics   HAProxy   HTML   Hacks   Hardware   Icinga   Influx   Internet   Java   KVM   Kibana   Kodi   Kubernetes   LVM   LXC   Linux   Logstash   Mac   Macintosh   Mail   MariaDB   Minio   MongoDB   Monitoring   Multimedia   MySQL   NFS   Nagios   Network   Nginx   OSSEC   OTRS   Office   OpenSearch   PGSQL   PHP   Perl   Personal   PostgreSQL   Postgres   PowerDNS   Proxmox   Proxy   Python   Rancher   Rant   Redis   Roundcube   SSL   Samba   Seafile   Security   Shell   SmartOS   Solaris   Surveillance   Systemd   TLS   Tomcat   Ubuntu   Unix   VMWare   VMware   Varnish   Virtualization   Windows   Wireless   Wordpress   Wyse   ZFS   Zoneminder