Automount USB disks in Ubuntu Server

I am using the usbmount package to automount a 1TB USB disk connected to my mini linux media server. This allows me to mount and unmount the external disk simply by turning it on and off, which is a convenient and power-saving arrangement.

Since I'm now storing DVD rips for my Popcorn Hour media player, I needed to add more storage to my streaming media server. My fit-PC takes 2.5" IDE/PATA drives, so it is already fully loaded with a 250G drive. So I got a 1TB 3.5" SATA drive (Western Digital GreenPower) and put it in a USB enclosure.

It's a very quiet, low power drive (about 9W generally), but I don't want it on 24/7. Spinning down the disk is one option, and I might investigate that further. But right now I want

1. To simply switch the disk on when I need it and off when I'm done.
2. For that to be safe (without potential for losing data).
3. To include the disk in the network share I export for the Popcorn Hour.

Desktops systems automount USB disks when you plug them in or power them on. This functionality is not available by default in Ubuntu server. (The packages used depend on the desktop environment.) Following a suggestion in the Ubuntu documentation, I installed the usbmount package

This package automatically mounts USB mass storage devices (typically USB pens) when they are plugged in, and unmounts them when they are removed. The mountpoints (/media/usb[0-7] by default), filesystem types to consider, and mount options are configurable. When multiple devices are plugged in, the first available mountpoint is automatically selected. If the device provides a model name, a symbolic link /var/run/usbmount/MODELNAME pointing to the mountpoint is automatically created. [...]

... which sounds ideal. And as far as I can see, it does the job.

I couldn't find much online about how to configure usbmount, but I soon found /etc/usbmount/usbmount.conf. You can configure mount options per filesystem type. The default options are

MOUNTOPTIONS="sync,noexec,nodev,noatime"

Out of the box, it only automounts ext2 and ext3 formatted volumes. I had formatted my new disk with XFS, so I added xfs to the configuration

FILESYSTEMS="ext2 ext3 xfs"

and it seems to work fine. When I switch on the disk, I see these messages in syslog:

kernel: [1034394.920087] usb 1-2: new high speed USB device using ehci_hcd and address 27
kernel: [1034395.064826] usb 1-2: configuration #1 chosen from 1 choice
kernel: [1034395.078845] scsi25 : SCSI emulation for USB Mass Storage devices
kernel: [1034395.085992] usb-storage: device found at 27
kernel: [1034395.086033] usb-storage: waiting for device to settle before scanning
kernel: [1034400.084595] usb-storage: device scan complete
kernel: [1034400.087474] scsi 25:0:0:0: Direct-Access     WDC WD10 EADS-00L5B1      0041 PQ: 0 ANSI: 0
kernel: [1034400.091636] sd 25:0:0:0: [sdb] 1953525168 512-byte hardware sectors (1000205 MB)
kernel: [1034400.094271] sd 25:0:0:0: [sdb] Write Protect is off
kernel: [1034400.094309] sd 25:0:0:0: [sdb] Mode Sense: 03 00 00 00
kernel: [1034400.094328] sd 25:0:0:0: [sdb] Assuming drive cache: write through
kernel: [1034400.104985] sd 25:0:0:0: [sdb] 1953525168 512-byte hardware sectors (1000205 MB)
kernel: [1034400.113058] sd 25:0:0:0: [sdb] Write Protect is off
kernel: [1034400.113094] sd 25:0:0:0: [sdb] Mode Sense: 03 00 00 00
kernel: [1034400.113114] sd 25:0:0:0: [sdb] Assuming drive cache: write through
kernel: [1034400.120193]  sdb: sdb1
kernel: [1034400.137434] sd 25:0:0:0: [sdb] Attached SCSI disk
kernel: [1034400.137980] sd 25:0:0:0: Attached scsi generic sg1 type 0
usbmount[16483]: executing command: mount -txfs -osync,noexec,nodev,noatime /dev/sdb1 /media/usb0
kernel: [1034401.228110] XFS mounting filesystem sdb1
kernel: [1034401.710070] Ending clean XFS mount for filesystem: sdb1
usbmount[16483]: executing command: run-parts /etc/usbmount/mount.d

The disk is mounted:

$ mount
<snip...>
/dev/sdb1 on /media/usb0 type xfs (rw,noexec,nodev,sync,noatime)

And when I switch it off, I see

kernel: [1034259.923629] usb 1-2: USB disconnect, address 26
kernel: [1034259.936251] xfs_force_shutdown(sdb1,0x1) called from line 420 of file /build/buildd/linux-2.6.27/fs/xfs/xfs_rw.c.  Return address = 0xe0a9ddb4
kernel: [1034259.936334] Filesystem "sdb1": I/O Error Detected.  Shutting down filesystem: sdb1
kernel: [1034259.936404] Please umount the filesystem, and rectify the problem(s)
kernel: [1034259.949818] Filesystem "sdb1": xfs_log_force: error 5 returned.
kernel: [1034259.949851] Filesystem "sdb1": xfs_log_force: error 5 returned.
kernel: [1034259.949890] xfs_force_shutdown(sdb1,0x1) called from line 420 of file /build/buildd/linux-2.6.27/fs/xfs/xfs_rw.c.  Return address = 0xe0a9ddb4
kernel: [1034259.968157] Filesystem "sdb1": xfs_log_force: error 5 returned.
usbmount[16307]: executing command: umount -l /media/usb0
kernel: [1034260.050924] Filesystem "sdb1": xfs_log_force: error 5 returned.
kernel: [1034260.057086] Filesystem "sdb1": xfs_log_force: error 5 returned.
kernel: [1034260.061208] xfs_force_shutdown(sdb1,0x1) called from line 420 of file /build/buildd/linux-2.6.27/fs/xfs/xfs_rw.c.  Return address = 0xe0a9ddb4
kernel: [1034260.069650] Filesystem "sdb1": xfs_log_force: error 5 returned.
kernel: [1034260.073675] Filesystem "sdb1": xfs_log_force: error 5 returned.
kernel: [1034260.077672] xfs_force_shutdown(sdb1,0x1) called from line 420 of file /build/buildd/linux-2.6.27/fs/xfs/xfs_rw.c.  Return address = 0xe0a9ddb4
usbmount[16307]: executing command: run-parts /etc/usbmount/umount.d
kernel: [1034260.084706] Filesystem "sdb1": xfs_log_force: error 5 returned.
kernel: [1034260.084735] Filesystem "sdb1": xfs_log_force: error 5 returned.
kernel: [1034260.084820] Filesystem "sdb1": xfs_log_force: error 5 returned.
kernel: [1034260.084864] Filesystem "sdb1": xfs_log_force: error 5 returned.
kernel: [1034260.084891] Filesystem "sdb1": xfs_log_force: error 5 returned.

which doesn't look so great. But after repeatedly mounting and unmounting the disk without any apparent problems, I was getting the feeling that this was not harmful.

I did a bit of searching to see whether XFS supports the sync mount option. The answer wasn't clear, so I raised a question on the XFS mailing list to see what they thought. The initial response was discouraging, but on followup it seems like since...

• I am sync mounting the filesystem (what usbmount does by default), and


• my kernel version is later than 2.6.17 (from which point write barrier support is enabled by default in XFS)

... I can keep on doing what I'm doing. I will not get corruption of the file system. (Do read the thread, the XFS FAQ and otherwise satisfy yourself. It's your data.)

The last item on my wish list was to include the disk in the network share I export for the Popcorn Hour. This is easy.

Usbmount mounts the disk to /media/usb[0-7] arbitrarily, but it also creates a symbolic link in /var/run/usbmount using the disk model name, which provides a consistent reference point. I created a link in my media storage directory to that point, e.g.

ln -s /home/shared/movies /var/run/usbmount/WDC_WD10_EADS-00L5B1_1/movies

Using my existing Samba share (/home/shared), this works perfectly. When the disk is on, I can browse into the linked directory from the Popcorn Hour. When the disk is off, the link is invisible.

Ubuntu USB stick install, part 2

I finally got a successful install. But it took several attempts and a lot of Google.

First attempt

In the fit-PC BIOS setup, I set Drive C: Nand Flash but neglected to set Drive D: IDE. So when the installer got round to the partitioning stage it could only see the USB stick, not the hard disk. Hit reset button and start again...

Second attempt

I didn't create a separate ext3 /boot partition, just XFS root and /home partitions. Apparently installing GRUB on an XFS partition is not a good thing. Near the end of the install, the installer offers to install LILO for me. Now I dimly remembered going through LILO versus GRUB before and preferring GRUB, so I try to install GRUB anyway. The installer warns me that this is a bad idea. When I ignore that this makes a fatal error in the install which then hangs. Grit teeth, hit reset button and start again...

Third and fourth attempts

I created a separate ext3 /boot partition to stay sweet with GRUB. So I now have

#1 primary 100MB ext3 /boot
#1 primary   4GB XFS  /
#5 logical 246GB XFS  /home

Now the install completed successfully each time. But it all ended in tears when I tried to reboot the fit-PC. Each time, it immediately dropped to a debugger interface, saying Embedded BIOS Debugger Breakpoint Trap and refused to do anything else. Weep, hit the sack and try again in the morning...

I noticed that after each install attempt, I could no longer boot from the USB stick. It gave GRUB Error 21. I could fix the stick by reconnecting it to my old Linux box and doing install-mbr /dev/sdb. But why?

I finally guessed that the Ubuntu installer must be installing GRUB to the MBR of my USB stick, instead of the hard disk in my fit-PC where I needed it. Armed with that hunch, Google finally gave me some answers, like this, this and this.

Fifth attempt

I let the Ubuntu installer run to the very end, where it tells you to reboot. At that point, I opened a console and took a look around.

I found (fdisk) that my hard disk partitions were mounted on /target. So I did

# chroot /target
# grub-install /dev/sdb

to install GRUB into the MBR of the hard disk. Then I went back to the installer's console and let it finish.

On restart I removed the USB stick, hit DEL to enter the BIOS setup, reverted to Drive C: IDE and saved to reboot.

On first attempt GRUB started (phew) but said

Error 21: Selected disk does not exist
Press any key to continue...

So I did. Then GRUB gives the menu list of image it can boot and allows you to edit and boot them. I edited the entry for the first image. It's first line was root (hd1,0). I edited that to root (hd0,0) and the hit 'b' to boot. The system booted at last.

Once in, I took a look at /boot/grub/menu.list. All three had root (hd1,0).

I rebooted to confirm that GRUB would fail again with same error.
It did. I fixed as before. Once back in, I edited /boot/grub/menu.list to change all the hd1 to hd0 and rebooted again.

At last, the system boots successfully. Now I can get on with the more interesting stuff.

Ubuntu USB stick install on fit-PC, part 1

Hard disk installed. Bootable USB stick ready. Here we go...

Following the advice on the fit-PC forum I made the following BIOS settings:

Drive C: Nand Flash | Boot 1st
Drive D: IDE

It booted to the installer from the USB stick. I chose the cli boot option and set it off.

So that's the USB stick part over then, I thought. Just a normal install from here. (Wrong! See part 2 for the gory details.)

The installer's first big question is partitioning. I went with

#1 primary   4GB XFS /
#5 logical 246GB XFS /home

No swap. I can add it later if I need it.

I'm giving XFS a try this time because I might end up storing some ISO files on this disk. It's supposed to be a good file system for large files.