RabbitMQ queue monitoring and fixing check_rabbitmq_queue monitoring plugin

Written by Claudio Kuenzler - 0 comments

Published on January 4th 2024 - Listed in Monitoring Coding Perl Linux

---

To monitor RabbitMQ queues, there are a couple of ways (scripts and monitoring plugins) but they all (usually) check the queues using the RabbitMQ API.

Viewing the queues from the API

The following example uses curl to access the RabbitMQ API and view all queues:

ck@rabbitmq:~$ curl http://localhost:15672/api/queues -u monitoring:secret
[{"arguments":{"x-dead-letter-exchange":"rabbitmq-prod-new-dlx"},"auto_delete":false,"backing_queue_status":{"avg_ack_egress_rate":0.01809491112502785,"avg_ack_ingress_rate":0.01809491112502785,"avg_egress_rate":0.01809491112502785,"avg_ingress_rate":0.01809491112502785,"delta":["delta",0,0,0,0],"len":0,"mode":"default","next_seq_id":4403,"q1":0,"q2":0,"q3":0,"q4":0,"target_ram_count":"infinity"},"consumer_capacity":1.0 [...]

The response is a (pretty large) JSON output, containing all queues including their statistics.

From that JSON output the different queues can be listed using the "name" key. Using the jshon parser:

ck@rabbitmq:~$ curl -s http://localhost:15672/api/queues -u monitoring:secret | jshon -a -e name
"billrun.completion"
"billrun.worker"
"invoice.download"
"invoice.dunning"
"invoice.post"
"invoice.processor"
"invoice.worker"

Or if you prefer the jq parser:

ck@rabbitmq:~$ curl -s http://localhost:15672/api/queues -u monitoring:secret | jq -r ".[].name"
"billrun.completion"
"billrun.worker"
"invoice.download"
"invoice.dunning"
"invoice.post"
"invoice.processor"
"invoice.worker"

As root user you can also use the rabbitmqctl command to list the queues:

root@rabbitmq:~# rabbitmqctl list_queues
Timeout: 60.0 seconds ...
Listing queues for vhost / ...
name    messages
invoice.processor    0
invoice.dunning    0
billrun.completion    0
invoice.download    0
invoice.worker    0
billrun.worker    0

Queues monitoring with check_rabbitmq_queue.pl

There are a couple of RabbitMQ scripts or monitoring plugins available, but all of them seem to be dated and sadly haven't received updates in years. One of the plugins which still works with a recent RabbitMQ version (tested with 3.9) is the Perl script check_rabbitmq_queue. This script is part of a bundle of Perl scripts (nagios-plugins-rabbitmq), doing all kinds of checks on a RabbitMQ server.

ck@rabbitmq:~$ /usr/lib/nagios/plugins/check_rabbitmq_queue.pl -H localhost
RABBITMQ_QUEUE OK - All queues under the thresholds | messages=273.5294;; messages_ready=273.5294;; messages_unacknowledged=0.0000;; consumers=0.8824;;

The %2F issue

Under certain circumstances, the plugin might run into a problem, that an Access Refused error shows up:

ck@rabbitmq:~$ /usr/lib/nagios/plugins/check_rabbitmq_queue.pl -H localhost --port 15672 --username monitoring --password secret
RABBITMQ_QUEUE CRITICAL - Access Refused : http://localhost:15672/api/queues/%2F

The issue here is that the plugin tries to handle (potentially) multiple virtual hosts, the default vhost being "/". But inside the plugin code, the vhost name is going through a Perl function called uri_escape.

my $vhost=uri_escape($p->opts->vhost);

This function translates characters into a URL-compatible format. A space in the URL (" ") would become %20 for example. But in this case %2F is the code for the forward slash ("/"), which is an important part of the URL itself and should not be translated.

This situation can be improved and fixed by only doing uri_escape on vhost names not matching "/":

my $vhost='/';
$vhost.=uri_escape($p->opts->vhost) if $p->opts->vhost ne "/";

And later in the code the URL to check the queue(s) is adjusted to the new default $vhost value:

my $url = "";
if ($queue eq "all"){
    $url = sprintf("http%s://%s:%d/api/queues%s", ($p->opts->ssl ? "s" : ""), $hostname, $port, $vhost);
} else{
    $url = sprintf("http%s://%s:%d/api/queues%s/%s", ($p->opts->ssl ? "s" : ""), $hostname, $port, $vhost, $queue);
}

After adding this fix, the queues can be monitored again:

ck@rabbitmq:~$ /usr/lib/nagios/plugins/check_rabbitmq_queue.pl -H localhost --port 15672 --username monitoring --password secret
RABBITMQ_QUEUE OK - All queues under the thresholds | messages=0.8636;; messages_ready=0.8636;; messages_unacknowledged=0.0000;; consumers=0.5682;;

This code change was submitted as pull request to the upstream project. But given that it has been 6 years since the last commit, I doubt this will be merged.

As an alternative, you can download the check_rabbitmq_queue plugin with the %2F fix from the pull request branch.

Add a comment

Show form to leave a comment

Comments (newest first)

No comments yet.