Very slow VMware Perl SDK with newer Perl libwww-perl version (solved)

Written by Claudio Kuenzler - 8 comments

Published on June 21st 2016 - last updated on May 9th 2019 - Listed in VMware Linux Nagios Icinga Monitoring

---

For a new VMware and monitoring environment, I wanted to add the ESXi hosts to the monitoring. In the past years I successfully used the plugin "check_esx" from OP5 for this purpose. Meanwhile I had to learn that the plugin was renamed to "check_vmware_api" and that this plugin in turn had been forked at least twice.

Now there are the following plugins out there which do basically the same but slightly differ in development and code:

• check_vmware_api from OP5 : The original based on check_esx
• check_vmware_api from Achintha Gunasekara: Forked from the OP5 plugin but to support the newer Perl module Monitoring::Plugin. However in the meantime (after the fork happened), the OP5 plugin is now using Monitoring::Plugin as well (see this commit).
• check_vmware_esx from Martin Fuerstenau: Forked from the OP5 plugin but with the purpose of being "lighter" than the original. This plugin (at least the command definition) is already part of Icinga 2 as ITL plugin.
  

They all have one thing in common: They rely on the VMware Perl SDK in the background. And here starts the problem.

With all three plugins I experienced timeouts. When I showed a little bit more patience, I finally saw that it takes more than 3 minutes (!!!), to get some results:

$ time /tmp/check_vmware_esx.pl -H esx001 -u root -p secret --select=runtime
SOAP request error - possibly a protocol issue:


[...]

real    3m46.739s
user    0m0.199s
sys     0m0.022s

The most interesting part is the first line of the output: SOAP request error - possibly a protocol issue. After some research I stumbled across the following two pages:

• https://communities.vmware.com/message/2298661
  
• https://bclary.com/blog/2014/04/17/how-to-work-around-soap-request-protocol-error-with-vmware-vsphere-perl-sdk-5-5-0/
  

The first site is a discussion in the VMware community forums where the problem is identified in the SDK itself. Seems the SDK was programmed with an old version of libwww-perl. On newer systems (and my monitoring server runs Ubuntu 14.04) this causes problems.

On the second site there is actually a nice workaround presented where the author, Bob Clary, manually installed an older libwww-perl version (5.837) into a separate location and exported the path in shell scripts as a wrapper around the Perl SDK.

While the workaround seems like the way to go, I didn't want to build wrappers around the SDK (and therefore maintain this after every potential update). If I work around it, then right in the file which causes problems. Which is, in my case, /usr/share/perl/5.18/VMware/VICommon.pm.

Before I did anything, I wanted to verify once again and directly from the Perl SDK itself, that the problem is not from one of the plugins but from the SDK:

time /usr/lib/vmware-vcli/apps/vm/vminfo.pl --server esx001 --username root --password secret
SOAP request error - possibly a protocol issue:
 xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
 xmlns:xsd="http://www.w3.org/2001/XMLSchema"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
[...]

real    3m48.655s
user    0m0.184s
sys     0m0.028s

Here again it took more than 3 minutes until a result came back.

Now it's time to install the older libwww-perl version, as seen on the article from Bob Clary:

cd src/
mkdir libwww-perl
cd libwww-perl
wget https://github.com/libwww-perl/libwww-perl/archive/libwww-perl/5.837.tar.gz
tar -xzf 5.837.tar.gz
cd libwww-perl-libwww-perl-5.837/
perl Makefile.PL INSTALL_BASE=/opt/check_vmware_esx
make
make install

This installs the following Perl folder in /opt/check_vmware_esx/lib/perl5:

ll /opt/check_vmware_esx/lib/perl5/
total 96
drwxr-xr-x 2 root root  4096 Jun 21 10:14 Bundle
drwxr-xr-x 2 root root  4096 Jun 21 10:14 File
drwxr-xr-x 2 root root  4096 Jun 21 10:14 HTML
drwxr-xr-x 5 root root  4096 Jun 21 10:14 HTTP
drwxr-xr-x 5 root root  4096 Jun 21 10:14 LWP
-r--r--r-- 1 root root  9137 Sep 20  2010 lwpcook.pod
-r--r--r-- 1 root root 21343 Sep 20  2010 LWP.pm
-r--r--r-- 1 root root 25528 Sep 20  2010 lwptut.pod
drwxr-xr-x 3 root root  4096 Jun 21 10:14 Net
drwxr-xr-x 3 root root  4096 Jun 21 10:14 WWW
drwxr-xr-x 3 root root  4096 Jun 21 10:14 x86_64-linux-gnu-thread-multi

Now to the modification of /usr/share/perl/5.18/VMware/VICommon.pm (which was installed by the VMware Perl SDK installer). The begin of the file looks like this:

#
# Copyright 2006 VMware, Inc.  All rights reserved.
#

use 5.006001;
use strict;
use warnings;

use Carp qw(confess croak);
use XML::LibXML;
use LWP::UserAgent;
use LWP::ConnCache;
use HTTP::Request;
use HTTP::Headers;
use HTTP::Response;
use HTTP::Cookies;
use Data::Dumper;
[...]

To use the manually installed libwww-perl module (and not the default one installed from the Ubuntu repository), I simply added "use lib" right before the Perl version is defined:

#
# Copyright 2006 VMware, Inc.  All rights reserved.
#

use lib "/opt/check_vmware_esx/lib/perl5";

use 5.006001;
use strict;
use warnings;

use Carp qw(confess croak);
use XML::LibXML;
use LWP::UserAgent;
use LWP::ConnCache;
use HTTP::Request;
use HTTP::Headers;
use HTTP::Response;
use HTTP::Cookies;
use Data::Dumper;

Let's try the vminfo.pl command from above again:

time /usr/lib/vmware-vcli/apps/vm/vminfo.pl --server esx001 --username root --password secret

Information of Virtual Machine vm001

Name:            vm001
No. of CPU(s):           1
Memory Size:             4096
Virtual Disks:           1
Template:                0
vmPathName:              [esxz108_T2] vm001/vm001.vmx
Guest OS:                Microsoft Windows Server 2008 R2 (64-bit)
guestId:                 windows7Server64Guest
Host name:               vm001.local
IP Address:              10.10.20.157
VMware Tools:            VMware Tools is running and the version is current
Cpu usage:               39 MHz
Host memory usage:               4141 MB
Guest memory usage:              327 MB
Overall Status:          The entity is OK

Information of Virtual Machine vm002

[...]

real    0m1.498s
user    0m0.834s
sys     0m0.035s

Wow, what a change! All VM's running on esx001 were listed with detailed information. And all that in 1.498 seconds!

The Perl SDK seems fixed. Does that also apply on the plugins? I tried it with check_vmware_esx:

time /tmp/check_vmware_esx.pl -H esx001 -u root -p secret -S cpu --nosession
OK: CPU wait=424610.00 ms - CPU ready=97.00 ms - CPU usage=7.10%|'cpu_wait'=424610.00ms;;;; 'cpu_ready'=97.00ms;;;; 'cpu_usage'=7.10%;;;;

real    0m0.687s
user    0m0.351s
sys     0m0.029s

Mission accomplished!

Update May 9th 2019:
Looking for a howto for Ubuntu Xenial with a newer VMware Perl SDK? Check out Installing check_vmware_esx.pl and VMware Perl SDK 6.7 on Ubuntu 16.04 Xenial.

Add a comment

Show form to leave a comment

Comments (newest first)

Natxo Asenjo from wrote on Dec 3rd, 2016:

Updating centos 7 provoked this problem :-(

I wanted to point out that there is another option, because this has been fixed upstream (in libwww-perl). I used cpanm (yum install perl-App-cpanminus) and then run cpanm LWP --notest. This will install the latest libwww to /usr/local/share/perl5 if run as root, or you could install it anywhere you liked using the PERL5LIB variable.

---

ck from Switzerland wrote on Oct 5th, 2016:

Please read https://github.com/BaldMansMojo/check_vmware_esx/issues/91:

SDK 6 is a pretty little thing of bull..t. You don't need it because it's not interacting with version 5.x correctly. See some older (and partly closed issues). SDK 5.x is enough. I'm running CentOS 6.x (Cluster with Nagios) and there is no chance to get SDK 6.x up and running. But as mentioned it is not needed. All of my ESX 6.x are monitored with SDK 5.x. Runs perfectly.

---

lmigo from wrote on Oct 5th, 2016:

Hi @ll, for me it's not working also. I use vSphere Perl SDK 6.0.2 [3561779] and libwww-perl like your instructions on Ubuntu Xenial and VMware vSphere 6.0.0.update02-3620759 as ESXi hosts. I've still timeouts like
.

It may not be true that I have to use old vSphere Perl SDKs. Really? Experience?
Can someone recommend a vSphere Perl SDK version which is working with vSphere 6?

Thanks for help guys.

---

ck from Switzerland wrote on Sep 15th, 2016:

Fred, try also the older perl sdk versions from VMware. One of the plugin authors told me that the newer one causes problems, yet the older ones still work with newer ESXi releases (6+).

---

fred from Hamburg wrote on Sep 15th, 2016:

I've just setup a Xenial with the latest x64 Perl SDK (Update2).The first run which is using the VMware Perl runs quick and after then it takes ages.
1. real 0m0.564s
2. real 5m36.058s
i tried your suggestion but it doesnt fix my problem.

---

geotek from Berlin wrote on Aug 11th, 2016:

Thanks so much for this brilliant information, it saved my day!!

---

ck from Switzerland wrote on Jun 27th, 2016:

Hi AFanBoy. You're asking the wrong guy. I'm also just an end user of the plugin and the SDK. However you might want to check the datacenter and not a single ESXi host which the plugin allows with the -D parameter. For more information about usage of the check_vmware_api or check_vmware_esx plugins, check out the links mentioned in the article.

---

AFanBoy from Germany wrote on Jun 27th, 2016:

Hey,
im using the Kaspersky Plugin for Nagios and it works great but i had 2-3 things.
I have 200+ clients and get every month 2-3 +.
Its hard to add every host in the list. Can you program it, that the script check all, and when a host gets an error that the error hostname are critial displayed in nagios?
I dont understand also no france, its hard to understand the source
Thanllols

---