Raspberry Pi sd card issue after power outage (constant red led)

Written by - 0 comments

Published on - Listed in Hardware Linux


Last night we had a (announced) power outage at work. This morning everything seemed to work again except one thing: The Raspberry Pi 2 showing our monitoring status on the TV screen. Nooooo, we're blind!

The Raspberry Pi's LED was a constant red. No output on HDMI and also keyboard/mouse didn't show any connectivity. Even without any USB devices connected and power cable unplugged/replugged the Pi wouldn't boot. 

I came across a few pages with good information for troubleshooting booting issues:

My first fear was that the power outage caused a defect in the polyfuse and that I'd either have to replace it or wait some time (up until a couple of days) so it auto-resets itself. But a lot of the posts almost always blamed the SD card for boot issues. I didn't really think this could be the cause but I needed to make sure anyways.

So I inserted the micro sd card into my computer and voilĂ , there's definitely something bad:

[ 2882.284344] usb 2-3: USB disconnect, device number 2
[ 2902.054057] mmc0: new high speed SDHC card at address 59b4
[ 2902.060081] Driver 'mmcblk' needs updating - please use bus_type methods
[ 2902.060139] mmcblk0: mmc0:59b4 00000 7.35 GiB
[ 2902.063721]  mmcblk0: p1 p2 < p5 p6 > p3
[ 2904.227853] EXT4-fs (mmcblk0p3): recovery complete
[ 2904.235789] EXT4-fs (mmcblk0p3): mounted filesystem with ordered data mode. Opts: (null)
[ 2904.269231] FAT-fs (mmcblk0p5): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
[ 2906.018193] EXT4-fs (mmcblk0p6): 9 orphan inodes deleted
[ 2906.018196] EXT4-fs (mmcblk0p6): recovery complete
[ 2906.065988] EXT4-fs (mmcblk0p6): mounted filesystem with ordered data mode. Opts: (null)
[ 2907.605639] mmcblk0: error -110 transferring data, sector 6016528, nr 56, cmd response 0x900, card status 0xb00
[ 2907.605643] mmcblk0: retrying using single block read
[ 2909.153337] mmcblk0: error -110 transferring data, sector 1830160, nr 32, cmd response 0x900, card status 0xb00
[ 2909.153349] mmcblk0: retrying using single block read
[ 2915.557948] mmcblk0: error -110 transferring data, sector 10471632, nr 8, cmd response 0x900, card status 0xb00
[ 2915.557953] mmcblk0: retrying using single block read
[ 2916.088688] mmcblk0: error -110 transferring data, sector 10472016, nr 8, cmd response 0x900, card status 0xb00
[ 2916.088694] mmcblk0: retrying using single block read

As dmesg suggested, I launched a fsck on /dev/mmcblk0p5 and got this:

fsck /dev/mmcblk0p5
fsck from util-linux 2.20.1
fsck.fat 3.0.26 (2014-03-07)
0x25: Dirty bit is set. Fs was not properly unmounted and some data may be corrupt.
1) Remove dirty bit
2) No action
? 1
Leaving filesystem unchanged.
/dev/mmcblk0p5: 74 files, 9965/30651 clusters

So a plain fsck didn't really repair the file system (Leaving filesystem unchanged). On one of the pages mentioned above, the following options were mentioned: -trawl.
From the --help output of fsck.fat, trawl means:

  -t       test for bad clusters
  -r       interactively repair the filesystem
  -a       automatically repair the filesystem
  -w       write changes to disk immediately
  -l       list path names

OK, looks legit to me. Let's roll:

fsck.fat -trawl /dev/mmcblk0p5
fsck.fat 3.0.26 (2014-03-07)
0x25: Dirty bit is set. Fs was not properly unmounted and some data may be corrupt.
 Automatically removing dirty bit.
Checking file /boot0
Checking file /COPYING.linux (COPYIN~1.LIN)
Checking file /LICENCE.broadcom (LICENC~1.BRO)
Checking file /LICENSE.oracle (LICENS~1.ORA)
Checking file /bcm2708-rpi-b-plus.dtb (BCM270~1.DTB)
Checking file /bcm2708-rpi-b.dtb (BCM270~2.DTB)
Checking file /bcm2709-rpi-2-b.dtb (BCM270~3.DTB)
Checking file /bootcode.bin (BOOTCODE.BIN)
Checking file /cmdline.txt (CMDLINE.TXT)
Checking file /config.txt (CONFIG.TXT)
Checking file /fixup.dat (FIXUP.DAT)
Checking file /fixup_cd.dat (FIXUP_CD.DAT)
Checking file /fixup_db.dat (FIXUP_DB.DAT)
Checking file /fixup_x.dat (FIXUP_X.DAT)
Checking file /issue.txt (ISSUE.TXT)
Checking file /kernel.img (KERNEL.IMG)
Checking file /kernel7.img (KERNEL7.IMG)
Checking file /os_config.json (OS_CON~1.JSO)
Checking file /start.elf (START.ELF)
Checking file /start_cd.elf (START_CD.ELF)
Checking file /start_db.elf (START_DB.ELF)
Checking file /start_x.elf (START_X.ELF)
Checking file /overlays (OVERLAYS)
Checking file /bcm2708-rpi-cm.dtb (BCM270~4.DTB)
Checking file /overlays/.
Checking file /overlays/..
Checking file /overlays/hifiberry-amp-overlay.dtb (HIFIBE~1.DTB)
Checking file /overlays/tinylcd35-overlay.dtb (TINYLC~1.DTB)
Checking file /overlays/bmp085_i2c-sensor-overlay.dtb (BMP085~1.DTB)
Checking file /overlays/hifiberry-digi-overlay.dtb (HIFIBE~2.DTB)
Checking file /overlays/i2c-rtc-overlay.dtb (I2C-RT~1.DTB)
Checking file /overlays/ads7846-overlay.dtb (ADS784~1.DTB)
Checking file /overlays/rpi-display-overlay.dtb (RPI-DI~1.DTB)
Checking file /overlays/README
Checking file /overlays/hifiberry-dac-overlay.dtb (HIFIBE~3.DTB)
Checking file /overlays/at86rf233-overlay.dtb (AT86RF~1.DTB)
Checking file /overlays/hy28b-overlay.dtb (HY28B-~1.DTB)
Checking file /overlays/rpi-proto-overlay.dtb (RPI-PR~1.DTB)
Checking file /overlays/hy28a-overlay.dtb (HY28A-~1.DTB)
Checking file /overlays/dht11-overlay.dtb (DHT11-~1.DTB)
Checking file /overlays/mz61581-overlay.dtb (MZ6158~1.DTB)
Checking file /overlays/iqaudio-dacplus-overlay.dtb (IQAUDI~1.DTB)
Checking file /overlays/lirc-rpi-overlay.dtb (LIRC-R~1.DTB)
Checking file /overlays/w1-gpio-overlay.dtb (W1-GPI~1.DTB)
Checking file /overlays/pps-gpio-overlay.dtb (PPS-GP~1.DTB)
Checking file /overlays/mcp2515-can0-overlay.dtb (MCP251~1.DTB)
Checking file /overlays/rpi-dac-overlay.dtb (RPI-DA~1.DTB)
Checking file /overlays/piscreen-overlay.dtb (PISCRE~1.DTB)
Checking file /overlays/spi-bcm2835-overlay.dtb (SPI-BC~1.DTB)
Checking file /overlays/w1-gpio-pullup-overlay.dtb (W1-GPI~2.DTB)
Checking file /overlays/pitft28-resistive-overlay.dtb (PITFT2~1.DTB)
Checking file /overlays/enc28j60-overlay.dtb (ENC28J~1.DTB)
Checking file /overlays/iqaudio-dac-overlay.dtb (IQAUDI~2.DTB)
Checking file /overlays/hifiberry-dacplus-overlay.dtb (HIFIBE~4.DTB)
Checking file /overlays/gpio-poweroff-overlay.dtb (GPIO-P~1.DTB)
Checking file /overlays/i2c-gpio-overlay.dtb (I2C-GP~1.DTB)
Checking file /overlays/i2s-mmap-overlay.dtb (I2S-MM~1.DTB)
Checking file /overlays/mcp2515-can1-overlay.dtb (MCP251~2.DTB)
Checking file /overlays/mmc-overlay.dtb (MMC-OV~1.DTB)
Checking file /overlays/piscreen2r-overlay.dtb (PISCRE~2.DTB)
Checking file /overlays/pitft28-capacitive-overlay.dtb (PITFT2~2.DTB)
Checking file /overlays/pwm-2chan-overlay.dtb (PWM-2C~1.DTB)
Checking file /overlays/pwm-overlay.dtb (PWM-OV~1.DTB)
Checking file /overlays/raspidac3-overlay.dtb (RASPID~1.DTB)
Checking file /overlays/rpi-backlight-overlay.dtb (RPI-BA~1.DTB)
Checking file /overlays/rpi-ft5406-overlay.dtb (RPI-FT~1.DTB)
Checking file /overlays/rpi-sense-overlay.dtb (RPI-SE~1.DTB)
Checking file /overlays/sdhost-overlay.dtb (SDHOST~1.DTB)
Checking file /overlays/sdio-overlay.dtb (SDIO-O~1.DTB)
Checking file /overlays/smi-dev-overlay.dtb (SMI-DE~1.DTB)
Checking file /overlays/smi-nand-overlay.dtb (SMI-NA~1.DTB)
Checking file /overlays/smi-overlay.dtb (SMI-OV~1.DTB)
Checking file /overlays/spi-bcm2708-overlay.dtb (SPI-BC~2.DTB)
Checking file /overlays/spi-dma-overlay.dtb (SPI-DM~1.DTB)
Checking file /overlays/uart1-overlay.dtb (UART1-~1.DTB)
Checking file /overlays/vga666-overlay.dtb (VGA666~1.DTB)
Performing changes.
/dev/mmcblk0p5: 74 files, 9965/30651 clusters

Alright, another fsck please to see if the dirty bit was removed:

fsck /dev/mmcblk0p5
fsck from util-linux 2.20.1
fsck.fat 3.0.26 (2014-03-07)
/dev/mmcblk0p5: 74 files, 9965/30651 clusters

Yep, looks better.

Once I inserted the sd card into the Raspberry Pi again, I couldn't believe it - that thing booted again. So it really was the sd card to blame!


Add a comment

Show form to leave a comment

Comments (newest first)

No comments yet.

RSS feed

Blog Tags:

  AWS   Android   Ansible   Apache   Apple   Atlassian   BSD   Backup   Bash   Bluecoat   CMS   Chef   Cloud   Coding   Consul   Containers   CouchDB   DB   DNS   Database   Databases   Docker   ELK   Elasticsearch   Filebeat   FreeBSD   Galera   Git   GlusterFS   Grafana   Graphics   HAProxy   HTML   Hacks   Hardware   Icinga   Influx   Internet   Java   KVM   Kibana   Kodi   Kubernetes   LVM   LXC   Linux   Logstash   Mac   Macintosh   Mail   MariaDB   Minio   MongoDB   Monitoring   Multimedia   MySQL   NFS   Nagios   Network   Nginx   OSSEC   OTRS   Office   OpenSearch   PGSQL   PHP   Perl   Personal   PostgreSQL   Postgres   PowerDNS   Proxmox   Proxy   Python   Rancher   Rant   Redis   Roundcube   SSL   Samba   Seafile   Security   Shell   SmartOS   Solaris   Surveillance   Systemd   TLS   Tomcat   Ubuntu   Unix   VMWare   VMware   Varnish   Virtualization   Windows   Wireless   Wordpress   Wyse   ZFS   Zoneminder