In the previous article I wrote about issues with the Spamhaus DNSBL, when public DNS resolvers are used. As mentioned at the end of that post, not only mail servers (such as Postfix) are running into potential issues, also monitoring is affected.
If you are managing any mail server which sends out e-mails, you should regularly check whether these servers are listed on a RBL/DNSBL (Real-time Block Lists / DNS Block Lists). Of course this is an annoying task and is easily forgotten, however very important. And here we come to the great thing, which open source monitoring is: There's a plugin for everything.
check_rbl is a monitoring plugin, maintained by Matteo Corti, which can query one or douzens of DNSBLs (defined in a ini config file). But since Spamhaus changed their policy to respond with a bogus A record "127.255.255.254" as response when public DNS resolvers were used, the plugin now mistakes the 127.255.255.254 record as being listed on the Spamhaus block lists.
root@icinga2:/usr/lib/nagios/plugins# ./check_rbl -H 212.103.71.210 --extra-opts "rbl@rbl.ini" -t 60
CHECK_RBL CRITICAL - 212.103.71.210 BLACKLISTED on 6 servers of 82 (cbl.abuseat.org (127.255.255.254) pbl.spamhaus.org (127.255.255.254) sbl.spamhaus.org (127.255.255.254) sbl-xbl.spamhaus.org (127.255.255.254) zen.spamhaus.org (127.255.255.254) xbl.spamhaus.org (127.255.255.254)) | servers=6;0;0 time=0s;;
The plugin, written in Perl, uses the Net::DNS module to do DNS lookups/queries. To run a DNS query, a DNS resolver needs to be initiated (here using $res):
use Net::DNS;
my $res = Net::DNS::Resolver->new;
To launch a full DNS query on a domain, the query function can be used:
my $response = $res->query("www.claudiokuenzler.com");
Note: check_rbl uses another function to run multiple queries in parallel, but this query function is used for demonstration.
But $response cannot be used in combination with print, it would just show garbage data:
print $response;
-> Net::DNS::Packet=HASH(0x55d9e9b22e28)
This is because $response contains a hash as data value (see related article From strings to array to hashes and how to view the values in Perl), so we need to parse this correctly. To see how the hash looks like we can use the Data::Dumper module:
ck@mintp ~ $ cat /tmp/simpledns.pl
#!/usr/bin/perl
use Net::DNS;
use Data::Dumper qw(Dumper);
my $res = Net::DNS::Resolver->new;
my $response = $res->query("www.claudiokuenzler.com");
print Dumper \$response;
ck@mintp ~ $ perl /tmp/simpledns.pl
$VAR1 = \bless( {
'xedns' => bless( {
'owner' => bless( {
'label' => []
}, 'Net::DNS::DomainName1035' ),
'type' => 41
}, 'Net::DNS::RR::OPT' ),
'id' => 59494,
'additional' => [],
'answer' => [
bless( {
'owner' => bless( {
'label' => [],
'origin' => bless( {
'label' => [
'www',
'claudiokuenzler',
'com'
]
}, 'Net::DNS::DomainName' )
}, 'Net::DNS::DomainName1035' ),
'address' => '?gG?',
'type' => 1,
'rdlength' => 4,
'class' => 1,
'ttl' => 930
}, 'Net::DNS::RR::A' )
],
'status' => 33152,
'question' => [
bless( {
'qname' => bless( {
'label' => [
'www',
'claudiokuenzler',
'com'
]
}, 'Net::DNS::DomainName1035' ),
'qclass' => 1,
'qtype' => 1
}, 'Net::DNS::Question' )
],
'replysize' => 57,
'count' => [
1,
1,
0,
0
],
'replyfrom' => '127.0.0.53',
'authority' => []
}, 'Net::DNS::Packet' );
Looks overwhelming, doesn't it? However the part we're interested in is nested inside the answer array. Inside answer there's the "address" key - which right now still holds some weird value. To show the actual value, this needs to be run through the array (using foreach):
ck@mintp ~ $ cat /tmp/simpledns.pl
#!/usr/bin/perl
use Net::DNS;
my $res = Net::DNS::Resolver->new;
my $response = $res->query("www.claudiokuenzler.com");
foreach $record ($response->answer) {
print $record->type, " record: ";
print $record->address, "\n";
}
ck@mintp ~ $ perl /tmp/simpledns.pl
A record: 212.103.71.210
Here we go, we got the A record for www.claudiokuenzler.com!
But where did we get this DNS response from? This can be seen above in the DataDumper output:
'replyfrom' => '127.0.0.53',
We can also list the nameserver(s) by retrieving a list using the nameservers function:
ck@mintp ~ $ cat /tmp/simpledns.pl
#!/usr/bin/perl
use Net::DNS;
my $res = Net::DNS::Resolver->new;
my @nameserver = $res->nameservers();
my $response = $res->query("www.claudiokuenzler.com");
foreach $record ($response->answer) {
print $record->type, " record: ";
print $record->address, "\n";
}
print "We used this resolver: @nameserver\n";
ck@mintp ~ $ perl /tmp/simpledns.pl
A record: 212.103.71.210
We used this resolver: 127.0.0.53
We can see that 127.0.0.53 is used as nameserver. This IP is defined in the systems /etc/resolv.conf (here using systemd-resolved in the background):
ck@mintp ~ $ cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
# 127.0.0.53 is the systemd-resolved stub resolver.
# run "systemd-resolve --status" to see details about the actual nameservers.
nameserver 127.0.0.53
This means that Perl's Net::DNS uses the system's DNS nameserver, usually defined in /etc/resolv.conf.
With this information in mind, we can verify which nameservers are used on the Icinga server, running the check_rbl plugin.
root@icinga2:/usr/lib/nagios/plugins# ./check_rbl -H 212.103.71.210 --extra-opts "rbl@rbl.ini" -t 60
CHECK_RBL CRITICAL - 212.103.71.210 BLACKLISTED on 6 servers of 82 (cbl.abuseat.org (127.255.255.254) pbl.spamhaus.org (127.255.255.254) sbl.spamhaus.org (127.255.255.254) sbl-xbl.spamhaus.org (127.255.255.254) zen.spamhaus.org (127.255.255.254) xbl.spamhaus.org (127.255.255.254)) | servers=6;0;0 time=0s;;
root@icinga2:/usr/lib/nagios/plugins# cat /etc/resolv.conf
nameserver 1.1.1.1
nameserver 8.8.8.8
We're using the Cloudflare public DNS resolver 1.1.1.1 and Google's 8.8.8.8 as fallback. Surprise? Not really. That exactly explains why check_rbl shows the Spamhaus block lists as critical.
Of course we could tell this Icinga server to use different nameservers. But there's a better solution: We can patch the check_rbl plugin and allow a specific nameserver to be used as DNS resolver!
The Net::DNS module allows to set a specific nameserver during initiation (above with the $res variable). This can be defined by using the nameservers function once again (but not saved as list this time):
ck@mintp ~ $ cat /tmp/simpledns.pl
#!/usr/bin/perl
use Net::DNS;
use Data::Dumper qw(Dumper);
my $res = Net::DNS::Resolver->new;
$res->nameservers("208.67.222.222");
my @nameserver = $res->nameservers();
my $response = $res->query("www.claudiokuenzler.com");
#print Dumper \$response;
foreach $record ($response->answer) {
print $record->type, " record: ";
print $record->address, "\n";
}
print "We used this resolver: @nameserver\n";
ck@mintp ~ $ perl /tmp/simpledns.pl
A record: 212.103.71.210
We used this resolver: 208.67.222.222
A specific nameserver (208.67.222.222, a public DNS resolver from Cisco) was defined to be used as resolver. Using this knowledge, we can adjust the check_rbl plugin.
Of course we need to make this a config option as additional parameter (-n / --nameserver):
$options->arg(
spec => 'nameserver|n=s',
help => 'Use this nameserver IP as DNS resolver (only one IP supported)',
required => 0,
);
During the plugin's DNS resolver initiation (which happens in the plugin's init_dns_resolver function), an additional if condition checks whether or not the nameserver parameter was used. If yes, the given input is used as nameserver:
sub init_dns_resolver {
my $retries = shift;
my $res = Net::DNS::Resolver->new();
if ( $res->can('force_v4') ) {
$res->force_v4(1);
}
if ($retries) {
$res->retry($retries);
}
if (defined $options->nameserver) {
$res->nameservers( $options->nameserver );
}
my @nameservers = $res->nameservers();
debug("Using DNS Resolver: @nameservers");
return $res;
}
With this patched version of check_rbl, the plugin can now be run against a different nameserver instead of the fixed nameservers defined in the OS:
root@icinga2:/usr/lib/nagios/plugins# ./check_rbl -H 212.103.71.210 --extra-opts "rbl@rbl.ini" -t 60 -n 208.67.222.222
CHECK_RBL OK - 212.103.71.210 (vip210.infiniroot.net) BLACKLISTED on 0 servers of 82 | servers=0;0;0 time=2s;;
And voilà! We didn't run into the Spamhaus bogus response anymore and the plugin works again!
Of course this patch was sent as a pull request to the upstream project, check_rbl. It's very likely that the next version (1.6.5) of check_rbl will contain this feature.
No comments yet.
AWS Android Ansible Apache Apple Atlassian BSD Backup Bash Bluecoat CMS Chef Cloud Coding Consul Containers CouchDB DB DNS Database Databases Docker ELK Elasticsearch Filebeat FreeBSD Galera Git GlusterFS Grafana Graphics HAProxy HTML Hacks Hardware Icinga Influx Internet Java KVM Kibana Kodi Kubernetes LVM LXC Linux Logstash Mac Macintosh Mail MariaDB Minio MongoDB Monitoring Multimedia MySQL NFS Nagios Network Nginx OSSEC OTRS Office OpenSearch PGSQL PHP Perl Personal PostgreSQL Postgres PowerDNS Proxmox Proxy Python Rancher Rant Redis Roundcube SSL Samba Seafile Security Shell SmartOS Solaris Surveillance Systemd TLS Tomcat Ubuntu Unix VMWare VMware Varnish Virtualization Windows Wireless Wordpress Wyse ZFS Zoneminder