For a couple of weeks now I'm making tests on a newly installed server running with Debian Squeezy (the current stable version). Everything went well (as mostly with Debian systems) UNTIL the first Kernel update arrived.
When I tried to upgrade the Kernel (linux-image-2.6.32-5-amd64), the upgrade process failed badly and with no real indication why (except of dependency problems). See for yourself:
# aptitude upgrade
The following partially installed packages will be configured:
linux-image-2.6-amd64 linux-image-2.6.32-5-amd64
No packages will be installed, upgraded, or removed.
0 packages upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B of archives. After unpacking 0 B will be used.
Setting up linux-image-2.6.32-5-amd64 (2.6.32-35squeeze2) ...
Running depmod.
Running update-initramfs.
update-initramfs: Generating /boot/initrd.img-2.6.32-5-amd64
W: Possible missing firmware /lib/firmware/rtl_nic/rtl8168d-2.fw for module r8169
W: Possible missing firmware /lib/firmware/rtl_nic/rtl8168d-1.fw for module r8169
W: mdadm: /etc/mdadm/mdadm.conf defines no arrays.
Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 2.6.32-5-amd64 /boot/vmlinuz-2.6.32-5-amd64
run-parts: executing /etc/kernel/postinst.d/zz-lilo 2.6.32-5-amd64 /boot/vmlinuz-2.6.32-5-amd64
Warning: Not updating LILO; /etc/lilo.conf not found
run-parts: executing /etc/kernel/postinst.d/zz-update-grub 2.6.32-5-amd64 /boot/vmlinuz-2.6.32-5-amd64
Generating grub.cfg ...
/usr/sbin/grub-probe: error: no such disk.
run-parts: /etc/kernel/postinst.d/zz-update-grub exited with return code 1
Failed to process /etc/kernel/postinst.d at /var/lib/dpkg/info/linux-image-2.6.32-5-amd64.postinst line 799.
dpkg: error processing linux-image-2.6.32-5-amd64 (--configure):
subprocess installed post-installation script returned error exit status 2
configured to not write apport reports
dpkg: dependency problems prevent configuration of linux-image-2.6-amd64:
linux-image-2.6-amd64 depends on linux-image-2.6.32-5-amd64; however:
Package linux-image-2.6.32-5-amd64 is not configured yet.
dpkg: error processing linux-image-2.6-amd64 (--configure):
dependency problems - leaving unconfigured
configured to not write apport reports
Errors were encountered while processing:
linux-image-2.6.32-5-amd64
linux-image-2.6-amd64
E: Sub-process /usr/bin/dpkg returned an error code (1)
This problem has been bugging me for a couple of weeks now, but only today I got time to make some investigation.
Now the important error to look at in the output is the following one (it's not obvious I know, but that's the source of the problem):
/usr/sbin/grub-probe: error: no such disk.
Let's take a look at the device.map file in /boot/grub, shall we?
# cat /boot/grub/device.map
(hd0) /dev/disk/by-id/scsi-35000c5001eb254ab
(hd1) /dev/disk/by-id/scsi-35000c5001eb1d31a
Okay, that doesn't look bad, but do these disks really exist?
# ll /dev/disk/by-id | grep scsi-
lrwxrwxrwx 1 root root 9 Aug 15 10:35 scsi-SATA_ST3750528AS_5VP390F3 -> ../../sda
lrwxrwxrwx 1 root root 10 Aug 15 10:35 scsi-SATA_ST3750528AS_5VP390F3-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 Aug 15 10:35 scsi-SATA_ST3750528AS_5VP390F3-part2 -> ../../sda2
lrwxrwxrwx 1 root root 10 Aug 15 10:35 scsi-SATA_ST3750528AS_5VP390F3-part3 -> ../../sda3
lrwxrwxrwx 1 root root 10 Aug 15 10:35 scsi-SATA_ST3750528AS_5VP390F3-part4 -> ../../sda4
lrwxrwxrwx 1 root root 10 Aug 15 10:35 scsi-SATA_ST3750528AS_5VP390F3-part5 -> ../../sda5
lrwxrwxrwx 1 root root 10 Aug 15 10:35 scsi-SATA_ST3750528AS_5VP390F3-part6 -> ../../sda6
lrwxrwxrwx 1 root root 9 Aug 15 10:35 scsi-SATA_ST3750528AS_5VP3994D -> ../../sdb
lrwxrwxrwx 1 root root 10 Aug 15 10:35 scsi-SATA_ST3750528AS_5VP3994D-part1 -> ../../sdb1
lrwxrwxrwx 1 root root 10 Aug 15 10:35 scsi-SATA_ST3750528AS_5VP3994D-part2 -> ../../sdb2
lrwxrwxrwx 1 root root 10 Aug 15 10:35 scsi-SATA_ST3750528AS_5VP3994D-part3 -> ../../sdb3
lrwxrwxrwx 1 root root 10 Aug 15 10:35 scsi-SATA_ST3750528AS_5VP3994D-part4 -> ../../sdb4
lrwxrwxrwx 1 root root 10 Aug 15 10:35 scsi-SATA_ST3750528AS_5VP3994D-part5 -> ../../sdb5
lrwxrwxrwx 1 root root 10 Aug 15 10:35 scsi-SATA_ST3750528AS_5VP3994D-part6 -> ../../sdb6
Nope, doesn't look like the devices mentioned in device.map exist! Disks sda and sdb actually use completely different names!
Now we need to put the correct information into device.map (change the wrong scsi-id's by the correct ones):
# cat /boot/grub/device.map
(hd0) /dev/disk/by-id/scsi-SATA_ST3750528AS_5VP390F3
(hd1) /dev/disk/by-id/scsi-SATA_ST3750528AS_5VP3994D
Once we changed the device ID's, that's not it - we need to tell grub we updated the device map:
# update-grub
Generating grub.cfg ...
Found linux image: /boot/vmlinuz-2.6.32-5-amd64
Found initrd image: /boot/initrd.img-2.6.32-5-amd64
done
And now, that the disk devices are set correctly, we launch the initial upgrade command again (for the Kernel ugprade):
# aptitude upgrade
The following partially installed packages will be configured:
linux-image-2.6-amd64 linux-image-2.6.32-5-amd64
No packages will be installed, upgraded, or removed.
0 packages upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B of archives. After unpacking 0 B will be used.
Setting up linux-image-2.6.32-5-amd64 (2.6.32-35squeeze2) ...
Running depmod.
Running update-initramfs.
update-initramfs: Generating /boot/initrd.img-2.6.32-5-amd64
W: Possible missing firmware /lib/firmware/rtl_nic/rtl8168d-2.fw for module r8169
W: Possible missing firmware /lib/firmware/rtl_nic/rtl8168d-1.fw for module r8169
W: mdadm: /etc/mdadm/mdadm.conf defines no arrays.
Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 2.6.32-5-amd64 /boot/vmlinuz-2.6.32-5-amd64
run-parts: executing /etc/kernel/postinst.d/zz-lilo 2.6.32-5-amd64 /boot/vmlinuz-2.6.32-5-amd64
Warning: Not updating LILO; /etc/lilo.conf not found
run-parts: executing /etc/kernel/postinst.d/zz-update-grub 2.6.32-5-amd64 /boot/vmlinuz-2.6.32-5-amd64
Generating grub.cfg ...
Found linux image: /boot/vmlinuz-2.6.32-5-amd64
Found initrd image: /boot/initrd.img-2.6.32-5-amd64
done
Setting up linux-image-2.6-amd64 (2.6.32+29) ...
And yes, this time it worked!
One question remains open though: Why did the Debian installer insert completely wrong scsi device id's into device.map ?
Update October 13: After some investigation with the data center guys, the problem came from an installation image which was now fixed.
No comments yet.
AWS Android Ansible Apache Apple Atlassian BSD Backup Bash Bluecoat CMS Chef Cloud Coding Consul Containers CouchDB DB DNS Database Databases Docker ELK Elasticsearch Filebeat FreeBSD Galera Git GlusterFS Grafana Graphics HAProxy HTML Hacks Hardware Icinga Influx Internet Java KVM Kibana Kodi Kubernetes LVM LXC Linux Logstash Mac Macintosh Mail MariaDB Minio MongoDB Monitoring Multimedia MySQL NFS Nagios Network Nginx OSSEC OTRS Office OpenSearch PGSQL PHP Perl Personal PostgreSQL Postgres PowerDNS Proxmox Proxy Python Rancher Rant Redis Roundcube SSL Samba Seafile Security Shell SmartOS Solaris Surveillance Systemd TLS Tomcat Ubuntu Unix VMWare VMware Varnish Virtualization Windows Wireless Wordpress Wyse ZFS Zoneminder