Upgrade of VPS server (Xen) from Debian Lenny to Squeeze (MySQL problems)

Written by Claudio Kuenzler - 0 comments

Published on February 4th 2012 - Listed in Linux MySQL Database

---

As it was announced on December 6th 2011, support for Debian Lenny will end on February 6th 2012. This means, no future updates will be released for Lenny systems. Time to upgrade.

The affected machine is a virtual machine (VPS) running as a Xen guest. Installed services are Apache2, MySQL, Nagios (compiled from source), PHP (compiled from source), OTRS (from source).

Will everything go smooth? I was especially worried about the kernel part, as Xen guests have their kernel pushed from the Xen host. In the Debian Upgrade Guide it is mentioned to upgrade the kernel manually BEFORE having the automatic apt-get script updating udev...

I will show my upgrade steps, based on the official Debian upgrading steps, and also what I did in case of problems.

1. Stop services. Nothing to comment here...

2. Backup! This is strongly encouraged. I backuped everything into /home/backup. Why? As mentioned in the Debian documentation, the /home directories won't be touched by the upgrade script.
The backup of /etc/ is very important. Besides that applications and user data should be backed up as well (e.g. dump of database otrs in my case).

3. Verify that the system is up to date and that no packages are on hold. This would break the upgrade process.
You can verify this by launching aptitude in the interactive mode and hit the key 'g'. If you get the message 'No packages are scheduled to be installed, removed, or upgraded.', then you're good to go.
To check the system for held packages, launch these commands. If you don't get an output, that's good!

myvps:/home/backup# dpkg --get-selections | grep hold
myvps:/home/backup# aptitude search "~ahold

Important note: The package 'splashy' mustn't be installed! In case you have this installed, remove it with 'apt-get purge splashy'.

4. It's time to update the /etc/apt/sources.list file. Just in case, make a backup of this file.
Since Debian Squeeze, there are is no 'debian-volatile' option anymore. So if you have an entry with debian-volatile, you must remove it from your sources.list.
Now replace all 'lenny' strings by 'squeeze'.
In my case, here's how my sources.list looked BEFORE:

deb ftp://ftp.hosteurope.de/mirror/ftp.debian.org/debian lenny main contrib non-free
deb ftp://ftp.hosteurope.de/mirror/ftp.debian.org/debian-volatile lenny/volatile main contrib non-free
deb ftp://ftp.hosteurope.de/mirror/ftp.debian.org/debian-security lenny/updates main contrib non-free

... and AFTER the modification:

deb ftp://ftp.hosteurope.de/mirror/ftp.debian.org/debian squeeze main contrib non-free
deb ftp://ftp.hosteurope.de/mirror/ftp.debian.org/debian-security squeeze/updates main contrib non-free

5. Now to the serious part! The upgrade!
In previous Debian versions, it was always recommended to use aptitude for the update, since the upgrade Lenny to Squeeze, apt-get should be used.
First update the packages list with all the new versions, so the system knows what to upgrade, then launch a minimal upgrade:

myvps:/# apt-get update

myvps:/# apt-get upgrade

The next step should be a manual upgrade of the kernel (linux-image). In the documentation this step is recommended so that the new version of udev (which will come in the next step) is compatible with the new kernel. But since this is a virtual xen server, there is no such kernel, so I skipped the manual kernel upgrade.

6. Once the minimal upgrade is completed, the distribution upgrade (dist-upgrade) follows:

myvps:/# apt-get dist-upgrade

And during this process, the following message appeared:

You are currently upgrading udev to a version that is not compatible with the currently running kernel.

You MUST install a compatible kernel version (2.6.26 or newer) before upgrading, otherwise the system may become UNUSABLE. Packages with a name starting with "linux-image-2.6-" provide a kernel image usable with this new udev version.

If you choose to upgrade udev nevertheless, you should install a compatible kernel and reboot with that kernel as soon as possible.

Proceed with the udev upgrade despite the kernel incompatibility?

YES                     NO

And this is exactly what I was afraid of at the beginning....
First I tried to select No - but then the dist-upgrade process is stopped and the upgrade not completed. So I relaunched the dist-upgrade and answered this question with YES.

The upgrade process continued and did finish after a couple of minutes. There's only one way to know, if there isn't a problem with udev and the xen kernel: A reboot ;-)

The machine booted just fine and was back after a couple of seconds, without any warnings or errors concerning the kernel or udev! Great! So far...

7. After the successful upgrade and reboot, the installed services had to be checked and to my big surprise, MySQL didn't run anymore. It couldn't even be started anymore:

myvps:/# /etc/init.d/mysql start
myvps:/# /etc/init.d/mysql status

As one can see, there isn't any output after the commands. Something's strange... A manual check confirmed the 'strangeness':

myvps:/# /usr/bin/mysqld_safe
-bash: /usr/bin/mysqld_safe: No such file or directory

Seems the upgrade script didn't do something that well. I checked, what is still installed or if everything was gone, and I found that the previous MySQL 5.0 was still installed, combined with some newer packages of 5.1:

myvps:/# dpkg -l | grep mysql
ii  libdbd-mysql-perl                     4.016-1                      Perl5 database interface to the MySQL database
ii  libmysqlclient15-dev                  5.0.51a-24+lenny5            MySQL database development files
ii  libmysqlclient15off                   5.0.51a-24+lenny5            MySQL database client library
ii  libmysqlclient16                      5.1.49-3                     MySQL database client library
ii  mysql-client-5.0                      5.0.51a-24+lenny5            MySQL database client binaries
ii  mysql-common                          5.1.49-3                     MySQL database common files, e.g. /etc/mysql/my.cnf
rc  mysql-server-5.0                      5.0.51a-24+lenny5            MySQL database server binaries
rc  php5-mysql                            5.2.6.dfsg.1-1+lenny13       MySQL module for php5
ii  python-mysqldb                        1.2.2-10+b1                  A Python interface to MySQL

To solve that problem, I simply launched a new installation of mysql-server-5.1:

myvps:/# apt-get install mysql-server-5.1
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
  libio-compress-base-perl libts-0.0-0 libltdl3-dev g++-4.3 libdns58 libmpfr1ldbl libdirectfb-1.0-0 libtool libbind9-50 libstdc++6-4.3-dev
  libtest-deep-perl libisccfg50 tsconf libio-compress-zlib-perl libossp-uuid15
Use 'apt-get autoremove' to remove them.
The following extra packages will be installed:
  mysql-client-5.1 mysql-server-core-5.1
Suggested packages:
  tinyca
The following packages will be REMOVED:
  mysql-client-5.0
The following NEW packages will be installed:
  mysql-client-5.1 mysql-server-5.1 mysql-server-core-5.1
0 upgraded, 3 newly installed, 1 to remove and 0 not upgraded.
Need to get 16.7 MB/20.8 MB of archives.
After this operation, 33.2 MB of additional disk space will be used.
Do you want to continue [Y/n]? y
Get:1 ftp://ftp.hosteurope.de/mirror/ftp.debian.org/debian/ squeeze/main mysql-client-5.1 amd64 5.1.49-3 [10.1 MB]
Get:2 ftp://ftp.hosteurope.de/mirror/ftp.debian.org/debian/ squeeze/main mysql-server-5.1 amd64 5.1.49-3 [6598 kB]
Fetched 16.7 MB in 0s (29.8 MB/s)
Preconfiguring packages ...
(Reading database ... 58764 files and directories currently installed.)
Removing mysql-client-5.0 ...
Processing triggers for man-db ...
Selecting previously deselected package mysql-client-5.1.
(Reading database ... 58708 files and directories currently installed.)
Unpacking mysql-client-5.1 (from .../mysql-client-5.1_5.1.49-3_amd64.deb) ...
Selecting previously deselected package mysql-server-core-5.1.
Unpacking mysql-server-core-5.1 (from .../mysql-server-core-5.1_5.1.49-3_amd64.deb) ...
Selecting previously deselected package mysql-server-5.1.
Unpacking mysql-server-5.1 (from .../mysql-server-5.1_5.1.49-3_amd64.deb) ...
Stopping MySQL database server: mysqld.
Processing triggers for man-db ...
Setting up mysql-client-5.1 (5.1.49-3) ...
Setting up mysql-server-core-5.1 (5.1.49-3) ...
Setting up mysql-server-5.1 (5.1.49-3) ...
Installing new version of config file /etc/init.d/mysql ...
Stopping MySQL database server: mysqld.
Starting MySQL database server: mysqld.
Checking for corrupt, not cleanly closed and upgrade needing tables..

Once this was done, a quick check of the MySQL init script and everything was in order again:

myvps:/# /etc/init.d/mysql status
/usr/bin/mysqladmin  Ver 8.42 Distrib 5.1.49, for debian-linux-gnu on x86_64
Copyright 2000-2008 MySQL AB, 2008 Sun Microsystems, Inc.
This software comes with ABSOLUTELY NO WARRANTY. This is free software,
and you are welcome to modify and redistribute it under the GPL license

Server version          5.1.49-3-log
Protocol version        10
Connection              Localhost via UNIX socket
UNIX socket             /var/run/mysqld/mysqld.sock
Uptime:                 22 sec

Threads: 1  Questions: 997  Slow queries: 0  Opens: 1445  Flush tables: 2  Open tables: 64  Queries per second avg: 45.318.

So to summarize the whole experience: The upgrade script did a good job, even though this is a VPS server without a real kernel installation. The 'udev' incompability turned out to be nothing but a warning (thank god!). The only thing to put a question mark to was the upgrade of MySQL; something clearly didn't work there as it should have. But the tables were untouched, the application OTRS was working fine again once the manual installation of MySQL 5.1 was done.

Add a comment

Show form to leave a comment

Comments (newest first)

No comments yet.