Last night we had a (announced) power outage at work. This morning everything seemed to work again except one thing: The Raspberry Pi 2 showing our monitoring status on the TV screen. Nooooo, we're blind!
The Raspberry Pi's LED was a constant red. No output on HDMI and also keyboard/mouse didn't show any connectivity. Even without any USB devices connected and power cable unplugged/replugged the Pi wouldn't boot.
I came across a few pages with good information for troubleshooting booting issues:
My first fear was that the power outage caused a defect in the polyfuse and that I'd either have to replace it or wait some time (up until a couple of days) so it auto-resets itself. But a lot of the posts almost always blamed the SD card for boot issues. I didn't really think this could be the cause but I needed to make sure anyways.
So I inserted the micro sd card into my computer and voilĂ , there's definitely something bad:
[ 2882.284344] usb 2-3: USB disconnect, device number 2
[ 2902.054057] mmc0: new high speed SDHC card at address 59b4
[ 2902.060081] Driver 'mmcblk' needs updating - please use bus_type methods
[ 2902.060139] mmcblk0: mmc0:59b4 00000 7.35 GiB
[ 2902.063721] mmcblk0: p1 p2 < p5 p6 > p3
[ 2904.227853] EXT4-fs (mmcblk0p3): recovery complete
[ 2904.235789] EXT4-fs (mmcblk0p3): mounted filesystem with ordered data mode. Opts: (null)
[ 2904.269231] FAT-fs (mmcblk0p5): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
[ 2906.018193] EXT4-fs (mmcblk0p6): 9 orphan inodes deleted
[ 2906.018196] EXT4-fs (mmcblk0p6): recovery complete
[ 2906.065988] EXT4-fs (mmcblk0p6): mounted filesystem with ordered data mode. Opts: (null)
[ 2907.605639] mmcblk0: error -110 transferring data, sector 6016528, nr 56, cmd response 0x900, card status 0xb00
[ 2907.605643] mmcblk0: retrying using single block read
[ 2909.153337] mmcblk0: error -110 transferring data, sector 1830160, nr 32, cmd response 0x900, card status 0xb00
[ 2909.153349] mmcblk0: retrying using single block read
[ 2915.557948] mmcblk0: error -110 transferring data, sector 10471632, nr 8, cmd response 0x900, card status 0xb00
[ 2915.557953] mmcblk0: retrying using single block read
[ 2916.088688] mmcblk0: error -110 transferring data, sector 10472016, nr 8, cmd response 0x900, card status 0xb00
[ 2916.088694] mmcblk0: retrying using single block read
As dmesg suggested, I launched a fsck on /dev/mmcblk0p5 and got this:
fsck /dev/mmcblk0p5
fsck from util-linux 2.20.1
fsck.fat 3.0.26 (2014-03-07)
0x25: Dirty bit is set. Fs was not properly unmounted and some data may be corrupt.
1) Remove dirty bit
2) No action
? 1
Leaving filesystem unchanged.
/dev/mmcblk0p5: 74 files, 9965/30651 clusters
So a plain fsck didn't really repair the file system (Leaving filesystem unchanged). On one of the pages mentioned above, the following options were mentioned: -trawl.
From the --help output of fsck.fat, trawl means:
-t test for bad clusters
-r interactively repair the filesystem
-a automatically repair the filesystem
-w write changes to disk immediately
-l list path names
OK, looks legit to me. Let's roll:
fsck.fat -trawl /dev/mmcblk0p5
fsck.fat 3.0.26 (2014-03-07)
0x25: Dirty bit is set. Fs was not properly unmounted and some data may be corrupt.
Automatically removing dirty bit.
Checking file /boot0
Checking file /COPYING.linux (COPYIN~1.LIN)
Checking file /LICENCE.broadcom (LICENC~1.BRO)
Checking file /LICENSE.oracle (LICENS~1.ORA)
Checking file /bcm2708-rpi-b-plus.dtb (BCM270~1.DTB)
Checking file /bcm2708-rpi-b.dtb (BCM270~2.DTB)
Checking file /bcm2709-rpi-2-b.dtb (BCM270~3.DTB)
Checking file /bootcode.bin (BOOTCODE.BIN)
Checking file /cmdline.txt (CMDLINE.TXT)
Checking file /config.txt (CONFIG.TXT)
Checking file /fixup.dat (FIXUP.DAT)
Checking file /fixup_cd.dat (FIXUP_CD.DAT)
Checking file /fixup_db.dat (FIXUP_DB.DAT)
Checking file /fixup_x.dat (FIXUP_X.DAT)
Checking file /issue.txt (ISSUE.TXT)
Checking file /kernel.img (KERNEL.IMG)
Checking file /kernel7.img (KERNEL7.IMG)
Checking file /os_config.json (OS_CON~1.JSO)
Checking file /start.elf (START.ELF)
Checking file /start_cd.elf (START_CD.ELF)
Checking file /start_db.elf (START_DB.ELF)
Checking file /start_x.elf (START_X.ELF)
Checking file /overlays (OVERLAYS)
Checking file /bcm2708-rpi-cm.dtb (BCM270~4.DTB)
Checking file /overlays/.
Checking file /overlays/..
Checking file /overlays/hifiberry-amp-overlay.dtb (HIFIBE~1.DTB)
Checking file /overlays/tinylcd35-overlay.dtb (TINYLC~1.DTB)
Checking file /overlays/bmp085_i2c-sensor-overlay.dtb (BMP085~1.DTB)
Checking file /overlays/hifiberry-digi-overlay.dtb (HIFIBE~2.DTB)
Checking file /overlays/i2c-rtc-overlay.dtb (I2C-RT~1.DTB)
Checking file /overlays/ads7846-overlay.dtb (ADS784~1.DTB)
Checking file /overlays/rpi-display-overlay.dtb (RPI-DI~1.DTB)
Checking file /overlays/README
Checking file /overlays/hifiberry-dac-overlay.dtb (HIFIBE~3.DTB)
Checking file /overlays/at86rf233-overlay.dtb (AT86RF~1.DTB)
Checking file /overlays/hy28b-overlay.dtb (HY28B-~1.DTB)
Checking file /overlays/rpi-proto-overlay.dtb (RPI-PR~1.DTB)
Checking file /overlays/hy28a-overlay.dtb (HY28A-~1.DTB)
Checking file /overlays/dht11-overlay.dtb (DHT11-~1.DTB)
Checking file /overlays/mz61581-overlay.dtb (MZ6158~1.DTB)
Checking file /overlays/iqaudio-dacplus-overlay.dtb (IQAUDI~1.DTB)
Checking file /overlays/lirc-rpi-overlay.dtb (LIRC-R~1.DTB)
Checking file /overlays/w1-gpio-overlay.dtb (W1-GPI~1.DTB)
Checking file /overlays/pps-gpio-overlay.dtb (PPS-GP~1.DTB)
Checking file /overlays/mcp2515-can0-overlay.dtb (MCP251~1.DTB)
Checking file /overlays/rpi-dac-overlay.dtb (RPI-DA~1.DTB)
Checking file /overlays/piscreen-overlay.dtb (PISCRE~1.DTB)
Checking file /overlays/spi-bcm2835-overlay.dtb (SPI-BC~1.DTB)
Checking file /overlays/w1-gpio-pullup-overlay.dtb (W1-GPI~2.DTB)
Checking file /overlays/pitft28-resistive-overlay.dtb (PITFT2~1.DTB)
Checking file /overlays/enc28j60-overlay.dtb (ENC28J~1.DTB)
Checking file /overlays/iqaudio-dac-overlay.dtb (IQAUDI~2.DTB)
Checking file /overlays/hifiberry-dacplus-overlay.dtb (HIFIBE~4.DTB)
Checking file /overlays/gpio-poweroff-overlay.dtb (GPIO-P~1.DTB)
Checking file /overlays/i2c-gpio-overlay.dtb (I2C-GP~1.DTB)
Checking file /overlays/i2s-mmap-overlay.dtb (I2S-MM~1.DTB)
Checking file /overlays/mcp2515-can1-overlay.dtb (MCP251~2.DTB)
Checking file /overlays/mmc-overlay.dtb (MMC-OV~1.DTB)
Checking file /overlays/piscreen2r-overlay.dtb (PISCRE~2.DTB)
Checking file /overlays/pitft28-capacitive-overlay.dtb (PITFT2~2.DTB)
Checking file /overlays/pwm-2chan-overlay.dtb (PWM-2C~1.DTB)
Checking file /overlays/pwm-overlay.dtb (PWM-OV~1.DTB)
Checking file /overlays/raspidac3-overlay.dtb (RASPID~1.DTB)
Checking file /overlays/rpi-backlight-overlay.dtb (RPI-BA~1.DTB)
Checking file /overlays/rpi-ft5406-overlay.dtb (RPI-FT~1.DTB)
Checking file /overlays/rpi-sense-overlay.dtb (RPI-SE~1.DTB)
Checking file /overlays/sdhost-overlay.dtb (SDHOST~1.DTB)
Checking file /overlays/sdio-overlay.dtb (SDIO-O~1.DTB)
Checking file /overlays/smi-dev-overlay.dtb (SMI-DE~1.DTB)
Checking file /overlays/smi-nand-overlay.dtb (SMI-NA~1.DTB)
Checking file /overlays/smi-overlay.dtb (SMI-OV~1.DTB)
Checking file /overlays/spi-bcm2708-overlay.dtb (SPI-BC~2.DTB)
Checking file /overlays/spi-dma-overlay.dtb (SPI-DM~1.DTB)
Checking file /overlays/uart1-overlay.dtb (UART1-~1.DTB)
Checking file /overlays/vga666-overlay.dtb (VGA666~1.DTB)
Performing changes.
/dev/mmcblk0p5: 74 files, 9965/30651 clusters
Alright, another fsck please to see if the dirty bit was removed:
fsck /dev/mmcblk0p5
fsck from util-linux 2.20.1
fsck.fat 3.0.26 (2014-03-07)
/dev/mmcblk0p5: 74 files, 9965/30651 clusters
Yep, looks better.
Once I inserted the sd card into the Raspberry Pi again, I couldn't believe it - that thing booted again. So it really was the sd card to blame!
No comments yet.
AWS Android Ansible Apache Apple Atlassian BSD Backup Bash Bluecoat CMS Chef Cloud Coding Consul Containers CouchDB DB DNS Database Databases Docker ELK Elasticsearch Filebeat FreeBSD Galera Git GlusterFS Grafana Graphics HAProxy HTML Hacks Hardware Icinga Influx Internet Java KVM Kibana Kodi Kubernetes LVM LXC Linux Logstash Mac Macintosh Mail MariaDB Minio MongoDB Monitoring Multimedia MySQL NFS Nagios Network Nginx OSSEC OTRS Office OpenSearch PGSQL PHP Perl Personal PostgreSQL Postgres PowerDNS Proxmox Proxy Python Rancher Rant Redis Roundcube SSL Samba Seafile Security Shell SmartOS Solaris Surveillance Systemd TLS Tomcat Ubuntu Unix VMWare VMware Varnish Virtualization Windows Wireless Wordpress Wyse ZFS Zoneminder