check_nrpe 4.x and NSClient++ 5.x: (ssl_err != 5) Error - Could not complete SSL handshake

Written by Claudio Kuenzler - 6 comments

Published on August 20th 2021 - Listed in Monitoring Windows

---

After upgrading the operating system of a monitoring server, the nagios-nrpe-plugin package (and therefore the check_nrpe plugin) was also updated. The current NRPE version on Ubuntu 20.04 (Focal) is 4.0.0:

root@focal:~# dpkg -l|grep nrpe
ii  nagios-nrpe-plugin                   4.0.0-2ubuntu1                    amd64        Nagios Remote Plugin Executor Plugin
ii  nagios-nrpe-server                   4.0.0-2ubuntu1                    amd64        Nagios Remote Plugin Executor Server

Shortly after the upgrade Windows hosts, which are monitored using NSClient++ (nscp), started to alert. It seems that the communication between check_nrpe and NSClient++ stopped working:

This can also be confirmed on the command line:

root@focal:~# /usr/lib/nagios/plugins/check_nrpe -H 10.10.22.23
CHECK_NRPE: (ssl_err != 5) Error - Could not complete SSL handshake with 10.10.22.23: 1

Even forcing the older NRPE protocol (v2) results in the same error:

root@focal:~# /usr/lib/nagios/plugins/check_nrpe -H 10.10.22.23 -2
CHECK_NRPE: (ssl_err != 5) Error - Could not complete SSL handshake with 10.10.22.23: 1

Communication still works with check_nrpe 2.15

The communication however still works with the old check_nrpe version (2.15), which was restored as nrpe2:

root@focal:~# /usr/lib/nagios/plugins/check_nrpe2 --help | head

NRPE Plugin for Nagios
Copyright (c) 1999-2008 Ethan Galstad (nagios@nagios.org)
Version: 2.15
Last Modified: 09-06-2013
License: GPL v2 with exemptions (-l for more info)
SSL/TLS Available: Anonymous DH Mode, OpenSSL 0.9.6 or higher required

Usage: check_nrpe -H <host> [ -b <bindaddr> ] [-4] [-6] [-n] [-u] [-p <port>] [-t <timeout>] [-c <command>] [-a <arglist...>]

root@focal:~# /usr/lib/nagios/plugins/check_nrpe2 -H 10.10.22.23
I (0.4.4.19 2015-12-08) seem to be doing fine...

So what did change in between?

NRPE was revived

NRPE itself staid at version 2.15 for many many years. This introduced a lot of weaknesses and security concerns. In 2018 NRPE was finally picked up again by Nagios Enterprises and until this year, 2021, new NRPE versions were released.

The major changes between the NRPE 2.x and the newer NRPE 3.x and 4.x concern the TLS/SSL settings. This also added a newer DH (Diffie-Hellman) key with a larger key size. From the README:

Running ./configure will now create a 2048-bit DH key instead of the old 512-bit key.

But how does that affect NSClient++?

NSClient++ uses smaller DH key

Thanks to this excellent blog post by Milan Kozak, one can find out that NSClient++ actually uses a smaller DH key. This can be seen inside the NSClient++ installation path, in the "security" folder:

NSClient++ itself uses an internal nrpe_dh_512.pem DH key, obviously still referring to the old NRPE version.

To be able to communicate with the newer check_nrpe version, a new DH key with a size of 2048 bytes (same as the check_nrpe plugin itself) needs to be created.

Create new DH key and configure NSClient++

The new DH key can easily be created on the monitoring server using the openssl command:

root@focal:~# openssl dhparam -C 2048 2> /dev/null|sed -n '/BEGIN/,/END/p'
-----BEGIN DH PARAMETERS-----
MIIBXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXwIBAg==
-----END DH PARAMETERS-----

Note: This command might take a few minutes. On a busy machine this will be generated faster.

The final output can be copied (including the BEGIN and END lines) and saved into a new file on the Windows server: C:\Program Files\NSClient++\security\nrpe_dh_2048.pem.

Now the configuration of NSClient++ (usually C:\Program Files\NSClient++\nsclient.ini) needs to be adjusted. Inside the NRPE server settings, add the parameter for the dh key:

; Section for NRPE (NRPEServer.dll) (check_nrpe) protocol options.
[/settings/NRPE/server]
; ENABLE SSL ENCRYPTION - This option controls if SSL should be enabled
use ssl = 1
; COMMAND ALLOW NASTY META CHARS - This option determines whether or not that we will allow clients to specify nasty (as in |`&><'"\[]{}) characters in arguments.
allow nasty characters = 1
; COMMAND ARGUMENT PROCESSING - This option determines whether or not that we will allow clients to specify arguments to commands that are executed.
allow arguments = 1
; PORT NUMBER - Port to use for NRPE.
port = 5666
; EXTENDED RESPONSE - Send more than 1 return packet to allow response to go beyond payload (requires modified client, if legacy is true this defaults to false).
extended response = 1
; # DH KEY
dh = ${certificate-path}/nrpe_dh_2048.pem

The ${certificate-path} variable is an internal NSClient++ variable which points to the security folder.

Now restart the NSClient++ service in services.msc:

Again Kudos to Milan Kozak for finding and sharing this solution!

Invalid packet version received from server

As soon as NSClient++ was re-configured to use the larger DH key and the service restarted, the check_nrpe plugin was able to communicate with this Windows host again.

However another warning started to show up:

root@focal:~# /usr/lib/nagios/plugins/check_nrpe -H 10.10.22.23
CHECK_NRPE: Invalid packet version received from server.
I (0.5.2.35 2018-01-28) seem to be doing fine...

The newer check_nrpe plugin uses the newer NRPE v3 protocol - but it seems that the NSClient++ agent answers with the older NRPE v2 protocol. To handle this, simply tell the check_nrpe plugin to use the older v2 protocol using the optional -2 parameter:

root@focal:~# /usr/lib/nagios/plugins/check_nrpe -H 10.10.22.23 -2
I (0.5.2.35 2018-01-28) seem to be doing fine...

Add a comment

Show form to leave a comment

Comments (newest first)

Nicolas from Santiago de Chile wrote on Sep 12th, 2024:

Efectivo, al colocarle la opción -2 luego de la IP, no mostro el mensaje "Invalid packet version"
Gracias

---

CK from Switzerland wrote on Aug 9th, 2024:

Hi Houssem,

> Can you clarify if this verify mode option is necessary for encryption and how the DH key is used for encryption on its own?

No, this verify option is not necessary for encrypting the NRPE communication. I have just verified with the NSClient++ NRPE settings mentioned above in the blog post. On the monitoring server, check_nrpe is executed this way:

ck@icinga:~$ /usr/lib/nagios/plugins/check_nrpe -H windowshost -P 8192 -2 -c check_drivesize
WARNING D:\: 287.699GB/349.872GB used, C:\: 206.648GB/249.509GB used|'D:\ used'=287.69882GB;279.89765;314.88486;0;349.87206 'D:\ used %'=82%;80;90;0;100 'C:\ used'=206.64754GB;199.60702;224.5579;0;249.50878 'C:\ used %'=83%;80;90;0;100 'X:\ used'=0B;0;0;0;0

At the same time I let a tcpdump run on the icinga server and everything was encrypted.

---

Houssem BB from wrote on Aug 8th, 2024:

My Centreon server is running on RHEL 8.9, and my host to be monitored via NSClient++ is Windows Server 2019. I don’t want to manage certificates, but I still want to benefit from encryption using the DH key. In my case, even the default 512-bit key generated during the NSClient++ installation on Windows does not trigger the SSL handshake error. This error only occurs when I enable the verify mode = peer-cert option.
The error in the NSClient logs is:
error: D:\a\nscp\nscp\include\socket/connection.hpp:276: Failed to establish secure connection: peer did not return a certificate (SSL routines, tls_process_client_certificate): 199.

So, to summarize, regardless of whether the default 512-bit key or a 2048-bit key is used, everything works as long as the verify mode option is commented out. The issue only arises when I enable the verify mode = peer-cert option.

This leads me to my question about the necessity of this option for enabling encryption with the DH key. Initially, I thought that the DH key was used to encrypt traffic with SSL certificates. However, I’m unclear on how this key alone is utilized by NSClient++ to encrypt packets without certificates. Can you clarify if this verify mode option is necessary for encryption and how the DH key is used for encryption on its own?

---

CK from Switzerland wrote on Aug 8th, 2024:

Hello Houssem. You can verify this with a tcpdump on the Linux side or Wireshark on the Windows side and should be able to see that the communication is encrypted (you should not be able to see any cleartext in the tcpdump).

> would enabling verify_mode = peer-cert with certificates provide significant additional security?

Yes, it would definitely add additional security as the connection would not be allowed without the certificate validation. But that is up to you whether or not this is considered a must (because it generates more maintenance work). In internal only LAN, maybe not so much, over Internet I would definitely recommend this.

---

Houssem BB from wrote on Aug 8th, 2024:

Hello everyone,

I am configuring NRPE to secure communications between Centreon and a remote host using SSL/TLS.
Currently, my nrpe section configuration is as follows:


=====================[ NRPE Server Settings ]====================
[/settings/NRPE/server]
allow arguments = true
allow nasty characters = true
port = 5666
; SSL/TLS Options
use ssl = true
extended response = 1
payload length = 8192
ssl options = no-sslv2,no-sslv3
;verify mode = peer-cert
;cert file = ${certificate-path}/certificate.pem
;key file = ${certificate-path}/certificate.pem
;ca file = ${certificate-path}/ca.pem
allowed hosts = 10.2.1.146
insecure = false
timeout = 60
; # DH KEY
dh = ${certificate-path}/2048.pem
I have commented out the lines related to verify_mode, certificates (cert file, key file, ca file), and I want to know if the encryption of the communication is still guaranteed with only the 2048-bit DH key (dh = ${certificate-path}/2048.pem).

My question is:

Is the use of the 2048-bit DH key alone sufficient to ensure the security of communications between Centreon and the remote host, or is it necessary to enable and configure SSL/TLS certificates (cert file, key file, ca file) as well?
Does the 2048-bit DH key alone provide adequate security, or would enabling verify_mode = peer-cert with certificates provide significant additional security?
Thank you in advance for your advice and recommendations!

---

Prathamesh Hankare from wrote on Jul 11th, 2024:

I faced the same issue with the new NRPE plugin on Ubuntu 22.04 for which I had to create a 2048 bit DH key and edit the config file to use the new DH key. Since I had more than 1000+ windows servers I wanted to monitor, I created a PowerShell script that does the work for me to monitor all the windows server(Specifically Domain Controllers). Anyone interested in the script can check out my GitHub repo. https://github.com/Prathameshhankare/nsclientpp_dh-key-updater

---