Ben Laurie blathering

Fun With FreeBSD and gmirror

A while ago I moved a lot of my stuff from a very ancient box to a quite new one. For some reason the new one has three disks in it, and so we (that is the ultra-patient Lemon and me) decided to mirror two of them. Not really having need of a third enormous disk it was left spare for now (possibly this was unwise in retrospect).

Since I run FreeBSD on my server boxes, we used gmirror. Being adventurous, we also decided we were going to mirror the root partition – slightly nerve-wracking, because when FreeBSD boots, it boots from the (unmirrored) root partition. But the theory is this works fine with mirrored disks.

So, we had three disks, which FreeBSD saw as ad4 (ata2-master), ad5 (ata2-slave) and ad6(ata3-master). We figured that ad4 and ad6 should be the mirrors, since they are on different controllers. So that’s what we did and it all works fine.

Fast forward several months and its time to upgrade the kernel. We’re moving from FreeBSD 6.x to FreeBSD 7.x, so its slightly nerve-wracking, but I do what I always do, which is to build the system from source, following the time-honoured

1. `cd /usr/src' (or to the directory containing your source tree).
2. `make buildworld'
3. `make buildkernel KERNCONF=YOUR_KERNEL_HERE' (default is GENERIC).
4. `make installkernel KERNCONF=YOUR_KERNEL_HERE' (default is GENERIC).
[steps 3. & 4. can be combined by using the "kernel" target]
5. `reboot' (in single user mode: boot -s from the loader prompt).
6. `mergemaster -p'
7. `make installworld'
8. `make delete-old'
9. `mergemaster'
10. `reboot'

(btw, mergemaster -U is good medicine for step 9). Everything goes fine. Until I realise uname -a is still reporting we’re running FreeBSD 6.2! WTF?

Well, to make a long story short, for some reason the BIOS thinks ata2-slave(i.e. our “spare” disk!) is the “first” disk, and so this is what it boots off. Presumably during the build the system got installed on “disk 1”, whichever that happened to be (we didn’t actually do the base build ourselves). Then when mirroring was set up, our “disk 1” didn’t match the BIOSes, and confusion reigned.

The happy ending to this story, though, is that

  • You can run FreeBSD 7 userland on a FreeBSD 6.2 kernel!
  • Switching the BIOS to boot off “disk 2” (i.e. ata2-master) made everything work as it should

I am recording this episode not because I think it is very interesting but because I hope it’ll be useful to someone else.

1 Comment

  1. I am using on board intel raid/mirror, just upgraded couple of days ago from 6.3 to 7.0,
    everything went smoothly, after couple of quirks with ruby recompilation.. see my upgrade
    notes in Website link, it’s in Slovak, but understandable.

    Comment by dan — 4 Jul 2008 @ 10:57

RSS feed for comments on this post.

Sorry, the comment form is closed at this time.

Powered by WordPress