How to install from CF media/USB drive using the ''grub'' bootloader.
I had been searching around for quite a while, without success, for a method to set up an Ubuntu installer to boot directly from a motherboard mounted Compact Flash drive. My goal was to have an installer that was fully grub compatible. The following is a write-up of the procedure. It was originally used to install Feisty but has been tested with Gutsy and should be easily adaptable for use with future releases. The procedure has been tested using the CF socket and Crucial Compact Flash memory as well as with USB memory sticks (Kingston Data Traveler, 1GB and 2GB and Crucial 1GB).
The host system used to configure the installer comprised the following:
- Ubuntu Feisty Fawn.
- Advantech PCM9381 motherboard
- 1.5GHz Celeron M CPU
- 1.0 GB Ram
- 100GB IDE hard disk on IDE Primary Master.
- Integral Compact Flash socket on IDE Secondary Master.
- 3 USB 2.0 ports.
Before anything else, the problem of finding the hardware designation for the CF/USB is paramount. You must be absolutely sure about this before starting. If you get this wrong, you may wipe out your entire system. Carefully examine the contents of each drive before starting and keep notes if necessary about which drive is which. You should also be aware that this assignment may change if different stuff is plugged in, or, indeed if things are connected in a different order. Also, note that the drive designator WILL change when you boot from the newly configured device: it will always be "hd0" if it is the boot device!
CF Socket. Since this is wired as the IDE secondary master, I naively assumed that it would appear as hd2, given that I expected hd1 would be the unused IDE primary slave. I was wrong. The BIOS closes this "gap" and numbers the drives sequentially from 0 on up. Since this is the second drive the BIOS discovers, it appears as hd1.
USB Memory Stick. After my experience with the CF socket, I was expecting the USB Stick to show up as hd1, and it did.
You should be aware that Ubuntu pretends that all drives are SCSI drives. Since I have only one hard drive and either the CF drive or the USB drive is installed (never both at the same time), the CF/USB drive shows up as "sdb" in Ubuntu's file browser (BUT as "hd1" in grub, which references hardware names, not driver names - Alas, there's no rest for the wicked.) I also tried to use a multi-slot/all in one USB card reader - from Crucial: Pocket Reader, model 1CR-T3-U28. There are 4 drives associated with this reader (sdc, sdd, sde, and sdf). The CF appeared on sdd. While I could freely access this drive from Ubuntu, I could never get the BIOS to boot from the external CF reader. If anyone figures out how to get this device to boot, please add instructions.
Step By Step Instructions
In the instruction that follow, all of the terminal input/output is indented, while my comments are at the margin. Also, since the procedure works for either the CF disk or the USB stick, I will refer to them both at the same time using the moniker "CF/USB". For all of the following steps, you will need a root terminal. Start up the terminal application and enter:
sbesch@hairpin:~$ sudo su
root@hairpin:/home/sbesch# cd /
(My computer's name - hairpin - comes from the name of a characteristic loop of amino acids found in many proteins, not, as one might assume, from the common tonsorial object.)
Step one is to get a partition table and a file system installed on the CF/USB. As far as I know, it is essential that the default file system that ships on most CF/USB's (FATxx) must be replaced. I tried valiantly to get grub to boot from a FATxx partition. It just wouldn't work. Why is a mystery. Grub patched in the correct Stage1.5, wrote the MBR, wrote the Stage2 loader - all successfully. It just wasn't recognized as a bootable device by my BIOS. To repartition and install a file system, I used "fdisk" in the example. Since the CF/USB automounts when plugged in, you will need to unmount it first - just click on the drive icon that appears and select "unmount". Then run fdisk and have a look at the partition table of the drive (gparted is a lot easier if you have it installed. This is just more universal):
root@hairpin:/# fdisk /dev/sdb
Command (m for help): p
Disk /dev/sdb: 1039 MB, 1039933440 bytes
32 heads, 62 sectors/track, 1023 cylinders
Units = cylinders of 1984 * 512 = 1015808 bytes
Device Boot Start End Blocks Id System
sdb1 * 1 1023 1014785 83 Linux
This is the table from my already partitioned CF. If you already have such a partition, then you are done with fdisk and can just enter "q" to exit. Note that the Boot Flag is set, as indicated by the "*" under the boot column. It must be set. If it's not, make sure that you set it before quitting (see below). If the drive is new, then there will probably be a FAT partition which will need to be removed. Enter "d" to delete the partition. If there is only one partition, it will just delete it. If there is more than one you will be asked to choose:
Command (m for help): d
Selected partition 1
Then create the new partition using the "n" command:
Command (m for help): n
p primary partition (1-4)
Partition number (1-4): 1
First cylinder (1-1023, default 1): 1
Last cylinder or +size or +sizeM or +sizeK (1-1023, default 1023):
Using default value 1023
Once the tables are set up, use the "a" command to set the "Boot" flag on the first partition:
Command (m for help): a
Partition number (1-4): 1
Note that you can set up more than one partition if you like. In fact, I do this with the 2GB USB drives so that I can have a more universal (i.e., FAT) file system on the unused part of the drive. The installer files require about 800 MB, so on the USB drive I have a partition table like this:
Command (m for help): p
Disk sdb: 2041 MB, 2041577472 bytes
255 heads, 63 sectors/track, 248 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
sdb1 * 1 102 819283+ 83 Linux
sdb2 103 248 1172745 6 FAT16
Up to now, nothing we have done has had any effect on the real table. If you've made any mistakes, or if you just want to start over, use the "q" command to quit without writing the new table. Otherwise, when you are ready, use the "w" command to write the table and make it take effect:
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
That's it. The table is written and ready to go. The "w" command also exits fdisk.
Making the File System
Next, we need a file system. I'm going to use ext2/3 (if you use gparted, the file system can be created directly when the partition is set up):
root@hairpin:/# mkfs -t ext2 /dev/sdb1
mke2fs 1.40-WIP (14-Nov-2006)
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
126976 inodes, 253696 blocks
12684 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=260046848
8 block groups
32768 blocks per group, 32768 fragments per group
15872 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376
Writing inode tables: done
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 34 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
If you've set up a second partition, you should put the filesystem on it now using mkfs or any of the other file system utilities (parted, gparted, etc).
Mounting the Drive.
The next step is to mount the CF/USB drive so the necessary files can be copied onto it. Generally, simply removing the device and plugging it back in will force a mount. (If you're using gparted, its rather difficult to keep it from being mounted after almost any operation on the disk, so it's probably already mounted.) On the other hand, it's simple enough to mount it right from the command line. First, you need a mount point. (This can be removed later with "rm -rf Bootie"):
root@hairpin:/# cd /media
root@hairpin:/media# mkdir Bootie
root@hairpin:/media# mount -t ext3 /dev/sdb1 /media/Bootie
Copying Needed Files.
Change to the mounted media, make the needed directories and copy the grub loader files from your system disk:
root@hairpin:/media# cd Bootie
root@hairpin:/media/Bootie# mkdir -p boot/grub
root@hairpin:/media/Bootie# cp /boot/grub/* ./boot/grub
The next thing is to get a suitable kernel and the initial ram-disk file system. It turns out that we need to be a bit careful here. We can't just take these from our system disk or any install CD. The installers on the CD's are tightly bound to the CD as the source medium and the installer gets very confused if you try to run the it from anything but the CD. However, Ubuntu supplies a customized kernel/initrd just for the purpose of installing from things like the CF or the USB disk. You get them from here:
Download both initrd.gz and vmlinuz from this site and place them in the "boot" folder just created on the CF/USB disk. Next, we need to get an install CD. The vmlinuz/initrd we just downloaded will look for an install CD ".iso" file on the install media and use it if one is found. We just need to download one to the root of the CF/USB disk. Simply go to the Ubuntu site and download the alternate CD (ubuntu-7.04-alternate-i386.iso) and copy it onto /media/Bootie. I also add an "id.txt" file to /media/Bootie (the root of the CF/USB disk) which simply contains the text "Feisty Installer". The idea of this file is to identify the purpose of the disk and the OS version, as well as to provide a unique file for identifying the drive in grub (see below). The file structure on the CF should now look like this:
While this example is for "feisty", I've also tested it with "gutsy". You just have to get the "7.10" alternate cd "iso" file and the matching set of installer files from:
In fact, when new versions arrive, all you should have to do is change vmlinuz/initrd, copy in the new alternate "iso" file and your ready to go.
Next step is to set up the grub configuration file (menu.lst) located in /media/Bootie/boot/grub. MAKE SURE THAT YOU EDIT THE ONE ON THE CF/USB DISK!! Using gedit (or your favorite editor), open the file:
root@hairpin:/media/Bootie# gedit ./boot/grub/menu.lst &
Change the file so that it contains the following lines. Note that all the disk references are now set to the first partition on hd0 - this will in fact be the drive designation of the CF/USB when you boot from it:
default 0 timeout 3 title Install in Text Mode kernel (hd0,0)/boot/vmlinuz fb=false video=vga16:off root=/dev/ram0 ramdisk_size=13000 quiet initrd (hd0,0)/boot/initrd.gz title Install in Expert Text Mode kernel (hd0,0)/boot/vmlinuz fb=false video=vga16:off priority=low root=/dev/ram0 ramdisk_size=13000 quiet initrd (hd0,0)/boot/initrd.gz title Rescue a broken system kernel (hd0,0)/boot/vmlinuz rescue/enable=true fb=false video=vga16:off root=/dev/ram0 ramdisk_size=13000 quiet initrd (hd0,0)/boot/initrd.gz
Note that your browser may break up the kernel line. When you copy and paste them, make sure that the three directives (title, kernel and initrd) each land on one line (i.e., no embedded <cr>'s). The items in this menu are intended to allow you to choose between several options that are provided on the alternate CD. I have only tested the default installer ("Install in Text Mode") and cannot vouch for the functionality of either of the remaining 2 options.
When done, close the file and unmount the CF/USB disk:
root@hairpin:/media/Bootie# cd ../..
Note that it's important that you unmount the CF/USB disk before running grub. You may get away with it if you don't, but the documentation warns that this may cause "inconsistencies in the disk I/O" (whatever that means).
Installing the bootloader
Now run grub to install the boot loader on the CF/USB disk:
Probing devices to guess BIOS drives. This may take a long time.
You probably won't see the previous line (at least for long enough to read it). While in grub, the terminal screen is replaced with the grub shell (below), hiding the initial response of grub. You will see it later when you exit grub. I've tried to illustrate the major points with some of the most useful grub features. Comments appear as appropriate. What follows then, is the grub dialog.
[ Minimal BASH-like line editing is supported. For the first word, TAB lists possible command completions. Anywhere else TAB lists the possible completions of a device/filename. ESC at any time exits. ]
The first thing to do is to absolutely confirm the correct drive designation. This is where the "id.txt" file comes in handy - providing of course that you don't foolishly place an identical file in the root of your system drive! We'll use grub's find command to locate the file:
grub> find /id.txt
The response from grub - "(hd1,0)" - has just told us that the CF/USB disk partition that contains the installer is "(hd1,0)". It has also illustrated an important bit of syntax: drives are designated in parentheses as "(hdx,y)", where x is the drive number and y is the partition number. We'll use this info in the next 2 steps.
Set The grub root.
Next, we must set the grub "root". This is so that grub can make adjustments to the loader that it installs so that it will boot properly when the drive letters are later changed by the BIOS during the actual boot - remember, it's "(hd1,0)" now, but will be "(hd0,0)" at boot time. I've entered only root (hd<tab> to show how grub will list all available devices using <tab> completion. We don't want hd0! This is the main, 100GB drive on which Ubuntu is already installed. If we make this the root, then when our CF/USB drive boots it will try to find files on places that it shouldn't:
grub> root (hd
Possible disks are: hd0 hd1
Next, I add "1,<tab>" to the command to see what partitions I have available:
grub> root (hd1,
Possible partitions are:
Partition num: 0, Filesystem type is ext2fs, partition type 0x83
Partition num: 1, Filesystem type is fat, partition type 0x6
I'm working with my USB device here, so there are 2 partitions. I want partition 0 (note that sdb1 is partition 0!). In the following command, I choose partition 0 by completing the command with "0)"<cr>. You may have only one partition, in which case grub will complete the line for you if you type a <tab>. I choose partition 0 by completing the command with "0)"<cr>:
grub> root (hd1,0)
Setup and Install the grub Bootstrap Code.
Since I know that the grub loader files are on hd1,0 (I copied them onto the CF after all), next I'll let grub set itself up on hd1 (the CF/USB drive). We don't specify a partition here since we are setting up the boot loader in the Master Boot Record (MBR) which resides outside any partition:
grub> setup (hd1)
Checking if "/boot/grub/stage1" exists... yes
Checking if "/boot/grub/stage2" exists... yes
Checking if "/boot/grub/e2fs_stage1_5" exists... yes
Running "embed /boot/grub/e2fs_stage1_5 (hd1)"... 17 sectors are embedded.
Running "install /boot/grub/stage1 (hd1) (hd1)1+17 p (hd1,0)/boot/grub/stage2 /boot/grub/menu.lst"... succeeded
Notice what's happened. Grub has detected the filesystem type and patched the stage 2 loader with the stage 1.5 fragment. This teaches stage2 how to deal with the ext2 filesystem (that's the "embed" command). It has then copied the stage1 file to the MBR of the CF/USB disk and followed it with the (now patched) stage2. As far as grub is concerned, it's done. So now we just quit grub and return to the root command prompt.
Booting the ''CF/USB'' Drive
At this point, you should be able to boot from the newly created media. The trick now is how to instruct your BIOS to boot from the device. On the Advantech PCM9381, I need to select HDD-1 for the CF device and USB-HDD for the USB memory stick. Unfortunately, every BIOS is different. Sometimes, it is clear from the wording of the choices, other times it is necessary to try virtually every combination until you find one that works. Once you have gotten the BIOS set up to boot from the CF/USB disk, grub will start and present you with the menu that you installed in the "menu.lst" file. I choose the "Text Mode" installer. The rest is standard stuff.