Xen, LVM, and Grub

I ran into an interesting problem with my new server the other day.

Due to a hardware problem with an external USB hard drive, I had to reboot the XEN host.

I had to do the reboot remotely … so I couldn’t intervene if something went wrong.  Yes, you guessed it, something did go wrong.

When I finally got home I found the system trying to boot ‘bree’ … which is one of the XEN guest machines.

Initially I thought it had something to do with the way grub2 finds kernels to boot … I figured it was finding all the kernels and selecting the one that occurred first (alphabetically) as the default.

Turns out this wasn’t the case … since this machine uses old grub an not grub2.

What it was doing is booting the partition with a label of ‘/’ … the problem is, there were a number of partitions with that label.  The base partition (/dev/sda1) that the XEN host lives on and a bunch of logical volumes that house the XEN guests.

I’m sure it was looking at all the partitions, and logical volumes, that were labeled ‘/’ (logical volumes first) and selecting the first one it encountered (bree) and tried to boot that.

The solution was pretty easy … I changed the grub.conf file so that any references to ‘root=LABEL=/’ were changed to ‘root=/dev/sda1’.  This way it wouldn’t look at the logical volumes at all.

I’ve never been a big fan of volume labels … they’re great if the boot drive’s identifier is going to change … but, in the case of a XEN & LVM setup, it’s important to make sure they are unique.  Probably better to use UUID’s instead of label’s in this case.

I’m not worried about the XEN guest’s booting the wrong partition … because the guest is only aware of the logical partitions that have been allocated to it.

Leave a Reply

Your email address will not be published. Required fields are marked *