Icinga 2 fails to upgrade package to 2.12.4 on a recently upgraded Ubuntu 18.04 Bionic (from Xenial)

Written by Claudio Kuenzler - 0 comments

Published on May 28th 2021 - Listed in Icinga Monitoring Linux

---

In a recent article we already wrote about an Icinga 2 problem when upgrading Debian 9 (Stretch) to 10 (Buster) and Icinga packages would not be upgraded. A similar problem also happens on an upgraded Ubuntu 18.04 (Bionic) machine, when the previous system was 16.04 (Xenial).

Simple upgrade breaks Icinga 2

In this case, a standard apt-get upgrade was launched to install the newest packages on an Ubuntu 18.04 (Bionic). This machine was recently upgraded from Ubuntu 16.04 (Xenial) but has been running on Bionic for a while now.

During apt was running the upgrade, a newer Icinga 2 version was also downloaded and installed:

Unpacking icinga2 (2.12.4-1.bionic) over (2.12.1-1.xenial) ...
Preparing to unpack .../50-icinga2-bin_2.12.4-1.bionic_amd64.deb ...
Unpacking icinga2-bin (2.12.4-1.bionic) over (2.12.1-1.xenial) ...
Preparing to unpack .../51-icinga2-common_2.12.4-1.bionic_all.deb ...
Unpacking icinga2-common (2.12.4-1.bionic) over (2.12.1-1.xenial) ...
Preparing to unpack .../52-icinga2-doc_2.12.4-1.bionic_all.deb ...
Unpacking icinga2-doc (2.12.4-1.bionic) over (2.12.1-1.xenial) ...

Note that the previous package (2.12.1) was still a xenial package. This means that after the OS upgrade, the Icinga packages for Xenial still remained installed. Again, that's the exact same problem as mentioned in the mentioned article (Icinga 2 with MySQL IDO fails to start after Debian release upgrade). The reason is that apt handles the versioning numeric and alphabetical. With the same 2.12.1 versions available on both xenial and bionic, the letter x is considered "newer" as it comes later in the alphabet.

At the end of the upgrade process, apt returned an error:

Errors were encountered while processing:
 icinga2-common
 icinga2-bin
 icinga2
E: Sub-process /usr/bin/dpkg returned an error code (1)

Wrong libboost packages

The reason for this can be spotted, when checking the current status of the Icinga 2 daemon:

root@satellite:~# systemctl status icinga2
- icinga2.service - Icinga host/service/network monitoring system
   Loaded: loaded (/lib/systemd/system/icinga2.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/icinga2.service.d
           |-limits.conf
   Active: failed (Result: exit-code) since Fri 2021-05-28 11:33:17 CEST; 1min 13s ago
 Main PID: 11778 (code=exited, status=127)

May 28 11:33:17 satellite systemd[1]: Starting Icinga host/service/network monitoring system...
May 28 11:33:17 satellite icinga2[11778]: /usr/lib/x86_64-linux-gnu/icinga2/sbin/icinga2: symbol lookup error: /usr/lib/x86_64-linux-gnu/icinga2/sbin/icinga2: undefined symbol: _ZNK5boost16re_detail_10670031cpp_regex_traits_implementati
May 28 11:33:17 satellite systemd[1]: icinga2.service: Main process exited, code=exited, status=127/n/a
May 28 11:33:17 satellite systemd[1]: icinga2.service: Failed with result 'exit-code'.
May 28 11:33:17 satellite systemd[1]: Failed to start Icinga host/service/network monitoring system.

The symbol lookup error mentions boost in the error message. Let's take a closer look at the installed boost packages:

root@satellite:~# dpkg -l|grep libboost
ii  libboost-context1.67.0-icinga:amd64         1.67.0-13.1.xenial      amd64  provides a sort of cooperative multitasking on a single thread
ii  libboost-coroutine1.67.0-icinga:amd64       1.67.0-13.1.xenial      amd64  provides a sort of cooperative multitasking on a single thread
ii  libboost-filesystem1.58.0:amd64             1.58.0+dfsg-5ubuntu3.1  amd64  filesystem operations (portable paths, iteration over directories, etc) in C++
ii  libboost-filesystem1.65.1:amd64             1.65.1+dfsg-0ubuntu5    amd64  filesystem operations (portable paths, iteration over directories, etc) in C++
ii  libboost-filesystem1.67.0-icinga:amd64      1.67.0-13.1.xenial      amd64  filesystem operations (portable paths, iteration over directories, etc) in C++
ii  libboost-iostreams1.58.0:amd64              1.58.0+dfsg-5ubuntu3.1  amd64        Boost.Iostreams Library
ii  libboost-iostreams1.65.1:amd64              1.65.1+dfsg-0ubuntu5    amd64        Boost.Iostreams Library
ii  libboost-program-options1.67.0-icinga:amd64 1.67.0-13.1.xenial      amd64  program options library for C++
ii  libboost-regex1.67.0-icinga:amd64           1.67.0-13.1.xenial      amd64  regular expression library for C++
ii  libboost-system1.58.0:amd64                 1.58.0+dfsg-5ubuntu3.1  amd64  Operating system (e.g. diagnostics support) library
ii  libboost-system1.65.1:amd64                 1.65.1+dfsg-0ubuntu5    amd64  Operating system (e.g. diagnostics support) library
ii  libboost-system1.67.0-icinga:amd64          1.67.0-13.1.xenial      amd64  Operating system (e.g. diagnostics support) library
ii  libboost-thread1.67.0-icinga:amd64          1.67.0-13.1.xenial      amd64  portable C++ multi-threading

The libbost packages obviously coming from the Icinga installation (appended with -icinga) are still based on Xenial. No wonder the now upgraded Icinga 2 package would have troubles with a "wrong" libboost package.

As the libboost packages in the Icinga 2 repositories are still the same version (1.67.0-13.1) an upgrade of these packages won't do it. Instead they need to be downgraded. Remember - b(ionic) comes before x(enial):

root@satellite:~# apt-get install libboost-context1.67.0-icinga=1.67.0-13.1.bionic libboost-coroutine1.67.0-icinga=1.67.0-13.1.bionic libboost-program-options1.67.0-icinga=1.67.0-13.1.bionic libboost-regex1.67.0-icinga=1.67.0-13.1.bionic libboost-system1.67.0-icinga=1.67.0-13.1.bionic libboost-thread1.67.0-icinga=1.67.0-13.1.bionic
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages will be DOWNGRADED:
  libboost-context1.67.0-icinga libboost-coroutine1.67.0-icinga libboost-program-options1.67.0-icinga libboost-regex1.67.0-icinga libboost-system1.67.0-icinga libboost-thread1.67.0-icinga
0 upgraded, 0 newly installed, 6 downgraded, 0 to remove and 0 not upgraded.
3 not fully installed or removed.
Need to get 1,805 kB of archives.
After this operation, 16.4 kB disk space will be freed.
Do you want to continue? [Y/n] y
Get:1 http://packages.icinga.com/ubuntu icinga-bionic/main amd64 libboost-context1.67.0-icinga amd64 1.67.0-13.1.bionic [228 kB]
Get:2 http://packages.icinga.com/ubuntu icinga-bionic/main amd64 libboost-system1.67.0-icinga amd64 1.67.0-13.1.bionic [229 kB]
Get:3 http://packages.icinga.com/ubuntu icinga-bionic/main amd64 libboost-thread1.67.0-icinga amd64 1.67.0-13.1.bionic [267 kB]
Get:4 http://packages.icinga.com/ubuntu icinga-bionic/main amd64 libboost-coroutine1.67.0-icinga amd64 1.67.0-13.1.bionic [243 kB]
Get:5 http://packages.icinga.com/ubuntu icinga-bionic/main amd64 libboost-program-options1.67.0-icinga amd64 1.67.0-13.1.bionic [356 kB]
Get:6 http://packages.icinga.com/ubuntu icinga-bionic/main amd64 libboost-regex1.67.0-icinga amd64 1.67.0-13.1.bionic [482 kB]
Fetched 1,805 kB in 0s (11.0 MB/s)               
dpkg: warning: downgrading libboost-context1.67.0-icinga:amd64 from 1.67.0-13.1.xenial to 1.67.0-13.1.bionic
(Reading database ... 224253 files and directories currently installed.)
Preparing to unpack .../0-libboost-context1.67.0-icinga_1.67.0-13.1.bionic_amd64.deb ...
Unpacking libboost-context1.67.0-icinga:amd64 (1.67.0-13.1.bionic) over (1.67.0-13.1.xenial) ...
dpkg: warning: downgrading libboost-system1.67.0-icinga:amd64 from 1.67.0-13.1.xenial to 1.67.0-13.1.bionic
Preparing to unpack .../1-libboost-system1.67.0-icinga_1.67.0-13.1.bionic_amd64.deb ...
Unpacking libboost-system1.67.0-icinga:amd64 (1.67.0-13.1.bionic) over (1.67.0-13.1.xenial) ...
dpkg: warning: downgrading libboost-thread1.67.0-icinga:amd64 from 1.67.0-13.1.xenial to 1.67.0-13.1.bionic
Preparing to unpack .../2-libboost-thread1.67.0-icinga_1.67.0-13.1.bionic_amd64.deb ...
Unpacking libboost-thread1.67.0-icinga:amd64 (1.67.0-13.1.bionic) over (1.67.0-13.1.xenial) ...
dpkg: warning: downgrading libboost-coroutine1.67.0-icinga:amd64 from 1.67.0-13.1.xenial to 1.67.0-13.1.bionic
Preparing to unpack .../3-libboost-coroutine1.67.0-icinga_1.67.0-13.1.bionic_amd64.deb ...
Unpacking libboost-coroutine1.67.0-icinga:amd64 (1.67.0-13.1.bionic) over (1.67.0-13.1.xenial) ...
dpkg: warning: downgrading libboost-program-options1.67.0-icinga:amd64 from 1.67.0-13.1.xenial to 1.67.0-13.1.bionic
Preparing to unpack .../4-libboost-program-options1.67.0-icinga_1.67.0-13.1.bionic_amd64.deb ...
Unpacking libboost-program-options1.67.0-icinga:amd64 (1.67.0-13.1.bionic) over (1.67.0-13.1.xenial) ...
dpkg: warning: downgrading libboost-regex1.67.0-icinga:amd64 from 1.67.0-13.1.xenial to 1.67.0-13.1.bionic
Preparing to unpack .../5-libboost-regex1.67.0-icinga_1.67.0-13.1.bionic_amd64.deb ...
Unpacking libboost-regex1.67.0-icinga:amd64 (1.67.0-13.1.bionic) over (1.67.0-13.1.xenial) ...
Setting up icinga2-common (2.12.4-1.bionic) ...
Setting up libboost-context1.67.0-icinga:amd64 (1.67.0-13.1.bionic) ...
Setting up libboost-program-options1.67.0-icinga:amd64 (1.67.0-13.1.bionic) ...
Setting up libboost-regex1.67.0-icinga:amd64 (1.67.0-13.1.bionic) ...
Setting up libboost-system1.67.0-icinga:amd64 (1.67.0-13.1.bionic) ...
Setting up libboost-thread1.67.0-icinga:amd64 (1.67.0-13.1.bionic) ...
Setting up libboost-coroutine1.67.0-icinga:amd64 (1.67.0-13.1.bionic) ...
Setting up icinga2-bin (2.12.4-1.bionic) ...
Setting up icinga2 (2.12.4-1.bionic) ...
Processing triggers for libc-bin (2.27-3ubuntu1.4) ...
W: APT had planned for dpkg to do more than it reported back (36 vs 40).
   Affected packages: icinga2-common:amd64

After this the previous apt-get upgrade was launched again to see if there are still problems:

root@satellite:~# apt-get upgrade
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

And Icinga 2 is running again:

root@satellite:~# systemctl status icinga2
- icinga2.service - Icinga host/service/network monitoring system
   Loaded: loaded (/lib/systemd/system/icinga2.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/icinga2.service.d
           |-limits.conf
   Active: active (running) since Fri 2021-05-28 11:39:12 CEST; 40s ago
  Process: 7783 ExecStartPre=/usr/lib/icinga2/prepare-dirs /etc/default/icinga2 (code=exited, status=0/SUCCESS)
 Main PID: 7799 (icinga2)

When will this be solved?

Unfortunately the issue is nothing new and has been reported as an issue. For Ubuntu users, the problem shouldn't happen for a while, as Ubuntu just restarted at the beginning of the alphabet. But Debian users will have the same problem again when upgrading from Buster to Bullseye due to the alphabetical order of the Icinga packages. The definitive solution will be when Icinga's package contain the distribution release version number, not the code name.

Add a comment

Show form to leave a comment

Comments (newest first)

No comments yet.