Icinga 2: Advanced usage of arrays/dictionaries for monitoring of partitions

Written by Claudio Kuenzler - 2 comments

Published on August 21st 2015 - last updated on November 22nd 2021 - Listed in Nagios Icinga Monitoring

---

In the previous post (Using arrays in Icinga 2 custom attributes to monitor partitions) I wrote about how arrays can be used in custom attributes which are then used by apply rules.

Now I went a step further and created dictionaries (different sub-values in each array) in the custom attributes (now called custom variables). The goal in this case was to quickly define new warning and critical thresholds for each partition - if wanted.

To summarize the first post, the host object contained an array "vars.partitions" as a custom attribute which defined the partitions to be monitored:

object Host "linux-host" {
  import "generic-host"
  address = "192.168.1.45"
  vars.os = "Linux"
  # Define partitions of this host:
  vars.partitions = [ "/", "/var", "/srv" ]
}

An apply rule then read the array and applied a service object for each value of this array (therefore four different partition checks):

apply Service "Diskspace " for (partition in host.vars.partitions) {
  import "generic-service"
  check_command = "nrpe"
  vars.nrpe_command = "check_disk"
  vars.nrpe_arguments = [ "15%", "5%", partition ]

  assign where host.address && host.vars.os == "Linux"
}

Great so far. But there is one minor issue: All of these partitions are now using the fixed NRPE arguments 15% and 5% for the warning and critical thresholds. Wouldn't it be nice (if we were older... lalala) to define on the fly thresholds which overwrite the defaults? Yes it would...

Dictionaries (and a lot of try'n'err) to the rescue!

By rewriting the vars.partition array with values for each partition, additional parameters can be set:

object Host "linux-host" {
  import "generic-host"
  address = "192.168.1.45"
  vars.os = "Linux"
  # Define partitions of this host:
  vars.partitions.slash = { mountpoint = "/" }
  vars.partitions.slashtmp = { mountpoint = "/tmp" }
  vars.partitions.slashsrv = { mountpoint = "/srv", warn = "95%", crit = "90%" }
  vars.partitions.slashvar = { mountpoint = "/var" }
}

For each partition I created a custom attribute "vars.partitions.partitionname" which itself contains several variables.
The mountpoint variable is mandatory and as you can see only one partition (/srv) defines special warning and critical thresholds.

Note: If you ask why I added the word slash in every attribute name, just try it with "vars.partitions.var" and you will find out like me, that Icinga really doesn't like the word var appearing there...

[Update October 12th 2015: Please check out the comment of Powelleb below which takes my apply rule one step further (no need to create the slashXXX names!).]

For Icinga itself the "vars.partitions" attribute is still somewhat of an array, so I am still able to use one and the same apply rule for all partitions found in this array:

apply Service "Apply-Diskchecks" for (partition_name => config in host.vars.partitions) {
  import "generic-service"

  vars += config
  if (!vars.warn) { vars.warn = "15%" }
  if (!vars.crit) { vars.crit = "5%" }

  display_name = "Diskspace " + vars.mountpoint
  check_command = "nrpe"
  vars.nrpe_command = "check_disk"
  vars.nrpe_arguments = [ vars.warn, vars.crit, vars.mountpoint ]

  assign where host.address && host.vars.os == "Linux"
  ignore where host.vars.applyignore.partitions == true
}

OK, now it gets more complicated but hear me out, it's worth it!

First the definition of the apply rule which I named "Apply-Diskchecks" for each partition found in the array "host.vars.partitions"  (as defined in the Host object above).
The word "config" here seems to be a fixed alias in the dictionary usage which stands for the main value (or the array name); in this case slash, slashtmp, slashsrv and slashvar.
Now to something important: Each variable (vars) now uses the config as a prefix. Ergo if I use vars.mountpoint now it is in reality vars.partitions.slash.mountpoint if we're in the loop for the slash partition.
Right after that I define the default thresholds, if the thresholds were not set within the dictionary.
The display_name is a combination of the string "Diskspace" followed by the value of vars.mountpoint.
In vars.nrpe_arguments I am now submitting the dynamically created values of vars.warn, vars.crit and vars.mountpoint. Which means: If I didn't use special thresholds, the ones just defined a few lines above are applied. vars.mountpoint is a mandatory variable (as I wrote above), therefore this is coming from the custom attributes of the host object itself.

The thresholds and actual NRPE arguments can be checked and verified in the Icinga Classic UI (in newer Icingaweb2 as well):

As you see, the /srv partition contains the special thresholds defined in the host object. Success!

The NRPE check command on the server to be monitored looks like this by the way:

command[check_disk]=/usr/lib/nagios/plugins/check_disk -w $ARG1$ -c $ARG2$ -W $ARG1$ -K $ARG2$ -p $ARG3$

Add a comment

Show form to leave a comment

Comments (newest first)

ck from St. Gallen, Switzerland wrote on Oct 12th, 2015:

Hi Powelleb, nice catch, thanks!

---

Powelleb from USA wrote on Oct 10th, 2015:

great information. i took your post and came up with this solution to simplify needing to create names after the partitions....

object Host "linux-host" {
import "generic-host"
address = "192.168.1.45"
vars.os = "Linux"
# Define partitions of this host:
vars.partitions["/"] = {}
vars.partitions["/tmp"]= {}
vars.partitions["/srv"] = {warn = "95%", crit = "90%" }
vars.partitions["/var"] = {}
}

note the partition_name use....
apply Service "Apply-Diskchecks" for (partition_name => config in host.vars.partitions) {
import "generic-service"

vars += config
if (!vars.warn) { vars.warn = "15%" }
if (!vars.crit) { vars.crit = "5%" }

display_name = "Diskspace " + partition_name
check_command = "nrpe"
vars.nrpe_command = "check_disk"
vars.nrpe_arguments = [ vars.warn, vars.crit, partition_name ]

assign where host.address && host.vars.os == "Linux"
ignore where host.vars.applyignore.partitions == true
}


displayname and servicename same

apply Service "Diskspace " for (partition_name => config in host.vars.partitions) {
import "generic-service"

vars += config
if (!vars.warn) { vars.warn = "15%" }
if (!vars.crit) { vars.crit = "5%" }

check_command = "nrpe"
vars.nrpe_command = "check_disk"
vars.nrpe_arguments = [ vars.warn, vars.crit, partition_name ]

assign where host.address && host.vars.os == "Linux"
ignore where host.vars.applyignore.partitions == true
}

---