Tag/tag.png

Unsupported Version
This article applies to an unsupported version of Ubuntu. More info...

How to install from CF media/USB drive using the ''grub'' bootloader.

I had been searching around for quite a while, without success, for a method to set up an Ubuntu installer to boot directly from a motherboard mounted Compact Flash drive. My goal was to have an installer that was fully grub compatible. The following is a write-up of the procedure. It was originally used to install Feisty but has been tested with Gutsy and should be easily adaptable for use with future releases. The procedure has been tested using the CF socket and Crucial Compact Flash memory as well as with USB memory sticks (Kingston Data Traveler, 1GB and 2GB and Crucial 1GB).

Host System

The host system used to configure the installer comprised the following:

  • Ubuntu Feisty Fawn.
  • Advantech PCM9381 motherboard
  • 1.5GHz Celeron M CPU
  • 1.0 GB Ram
  • 100GB IDE hard disk on IDE Primary Master.
  • Integral Compact Flash socket on IDE Secondary Master.
  • 3 USB 2.0 ports.

Preliminaries

Before anything else, the problem of finding the hardware designation for the CF/USB is paramount. You must be absolutely sure about this before starting. If you get this wrong, you may wipe out your entire system. Carefully examine the contents of each drive before starting and keep notes if necessary about which drive is which. You should also be aware that this assignment may change if different stuff is plugged in, or, indeed if things are connected in a different order. Also, note that the drive designator WILL change when you boot from the newly configured device: it will always be "hd0" if it is the boot device!

  • CF Socket. Since this is wired as the IDE secondary master, I naively assumed that it would appear as hd2, given that I expected hd1 would be the unused IDE primary slave. I was wrong. The BIOS closes this "gap" and numbers the drives sequentially from 0 on up. Since this is the second drive the BIOS discovers, it appears as hd1.

  • USB Memory Stick. After my experience with the CF socket, I was expecting the USB Stick to show up as hd1, and it did.

You should be aware that Ubuntu pretends that all drives are SCSI drives. Since I have only one hard drive and either the CF drive or the USB drive is installed (never both at the same time), the CF/USB drive shows up as "sdb" in Ubuntu's file browser (BUT as "hd1" in grub, which references hardware names, not driver names - Alas, there's no rest for the wicked.) I also tried to use a multi-slot/all in one USB card reader - from Crucial: Pocket Reader, model 1CR-T3-U28. There are 4 drives associated with this reader (sdc, sdd, sde, and sdf). The CF appeared on sdd. While I could freely access this drive from Ubuntu, I could never get the BIOS to boot from the external CF reader. If anyone figures out how to get this device to boot, please add instructions.

Step By Step Instructions

In the instruction that follow, all of the terminal input/output is indented, while my comments are at the margin. Also, since the procedure works for either the CF disk or the USB stick, I will refer to them both at the same time using the moniker "CF/USB". For all of the following steps, you will need a root terminal. Start up the terminal application and enter:

  • sbesch@hairpin:~$ sudo su
    Password:Its_So_Obvious!
    root@hairpin:/home/sbesch# cd /
    root@hairpin:/#

(My computer's name - hairpin - comes from the name of a characteristic loop of amino acids found in many proteins, not, as one might assume, from the common tonsorial object.)

Partition Table.

Step one is to get a partition table and a file system installed on the CF/USB. As far as I know, it is essential that the default file system that ships on most CF/USB's (FATxx) must be replaced. I tried valiantly to get grub to boot from a FATxx partition. It just wouldn't work. Why is a mystery. Grub patched in the correct Stage1.5, wrote the MBR, wrote the Stage2 loader - all successfully. It just wasn't recognized as a bootable device by my BIOS. To repartition and install a file system, I used "fdisk" in the example. Since the CF/USB automounts when plugged in, you will need to unmount it first - just click on the drive icon that appears and select "unmount". Then run fdisk and have a look at the partition table of the drive (gparted is a lot easier if you have it installed. This is just more universal):

  • root@hairpin:/# fdisk /dev/sdb

    Command (m for help): p

    Disk /dev/sdb: 1039 MB, 1039933440 bytes
    32 heads, 62 sectors/track, 1023 cylinders
    Units = cylinders of 1984 * 512 = 1015808 bytes

    • Device Boot      Start         End      Blocks   Id  System
      sdb1   *           1          1023      1014785  83  Linux

This is the table from my already partitioned CF. If you already have such a partition, then you are done with fdisk and can just enter "q" to exit. Note that the Boot Flag is set, as indicated by the "*" under the boot column. It must be set. If it's not, make sure that you set it before quitting (see below). If the drive is new, then there will probably be a FAT partition which will need to be removed. Enter "d" to delete the partition. If there is only one partition, it will just delete it. If there is more than one you will be asked to choose:

  • Command (m for help): d
    Selected partition 1

Then create the new partition using the "n" command:

  • Command (m for help): n

    Command action
       e   extended
       p   primary partition (1-4)
    p
    Partition number (1-4): 1
    First cylinder (1-1023, default 1): 1
    Last cylinder or +size or +sizeM or +sizeK (1-1023, default 1023):
    Using default value 1023

Once the tables are set up, use the "a" command to set the "Boot" flag on the first partition:

  • Command (m for help): a
    Partition number (1-4): 1

Note that you can set up more than one partition if you like. In fact, I do this with the 2GB USB drives so that I can have a more universal (i.e., FAT) file system on the unused part of the drive. The installer files require about 800 MB, so on the USB drive I have a partition table like this:

  • Command (m for help): p

    Disk sdb: 2041 MB, 2041577472 bytes
    255 heads, 63 sectors/track, 248 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes

    Device Boot      Start         End      Blocks   Id  System
      sdb1   *           1         102      819283+  83  Linux
      sdb2             103         248     1172745    6  FAT16

Up to now, nothing we have done has had any effect on the real table. If you've made any mistakes, or if you just want to start over, use the "q" command to quit without writing the new table. Otherwise, when you are ready, use the "w" command to write the table and make it take effect:

  • Command (m for help): w

    The partition table has been altered!

    Calling ioctl() to re-read partition table.
    Syncing disks.
    root@hairpin:/#

That's it. The table is written and ready to go. The "w" command also exits fdisk.

Making the File System

Next, we need a file system. I'm going to use ext2/3 (if you use gparted, the file system can be created directly when the partition is set up):

  • root@hairpin:/# mkfs -t ext2 /dev/sdb1
    mke2fs 1.40-WIP (14-Nov-2006)
    Filesystem label=
    OS type: Linux
    Block size=4096 (log=2)
    Fragment size=4096 (log=2)
    126976 inodes, 253696 blocks
    12684 blocks (5.00%) reserved for the super user
    First data block=0
    Maximum filesystem blocks=260046848
    8 block groups
    32768 blocks per group, 32768 fragments per group
    15872 inodes per group
    Superblock backups stored on blocks: 
            32768, 98304, 163840, 229376

    Writing inode tables: done
    Creating journal (4096 blocks): done
    Writing superblocks and filesystem accounting information: done

    This filesystem will be automatically checked every 34 mounts or
    180 days, whichever comes first.  Use tune2fs -c or -i to override.

If you've set up a second partition, you should put the filesystem on it now using mkfs or any of the other file system utilities (parted, gparted, etc).

Mounting the Drive.

The next step is to mount the CF/USB drive so the necessary files can be copied onto it. Generally, simply removing the device and plugging it back in will force a mount. (If you're using gparted, its rather difficult to keep it from being mounted after almost any operation on the disk, so it's probably already mounted.) On the other hand, it's simple enough to mount it right from the command line. First, you need a mount point. (This can be removed later with "rm -rf Bootie"):

  • root@hairpin:/# cd /media
    root@hairpin:/media# mkdir Bootie
    root@hairpin:/media# mount -t ext3 /dev/sdb1 /media/Bootie

Copying Needed Files.

Change to the mounted media, make the needed directories and copy the grub loader files from your system disk:

  • root@hairpin:/media# cd Bootie
    root@hairpin:/media/Bootie# mkdir -p boot/grub
    root@hairpin:/media/Bootie# cp /boot/grub/* ./boot/grub

The next thing is to get a suitable kernel and the initial ram-disk file system. It turns out that we need to be a bit careful here. We can't just take these from our system disk or any install CD. The installers on the CD's are tightly bound to the CD as the source medium and the installer gets very confused if you try to run the it from anything but the CD. However, Ubuntu supplies a customized kernel/initrd just for the purpose of installing from things like the CF or the USB disk. You get them from here:

Download both initrd.gz and vmlinuz from this site and place them in the "boot" folder just created on the CF/USB disk. Next, we need to get an install CD. The vmlinuz/initrd we just downloaded will look for an install CD ".iso" file on the install media and use it if one is found. We just need to download one to the root of the CF/USB disk. Simply go to the Ubuntu site and download the alternate CD (ubuntu-7.04-alternate-i386.iso) and copy it onto /media/Bootie. I also add an "id.txt" file to /media/Bootie (the root of the CF/USB disk) which simply contains the text "Feisty Installer". The idea of this file is to identify the purpose of the disk and the OS version, as well as to provide a unique file for identifying the drive in grub (see below). The file structure on the CF should now look like this:

  • /
     ubuntu-7.04-alternate-i386.iso
     id.txt
     boot/
      initrd.gz
      vmlinuz
      grub/
       ....loader files

While this example is for "feisty", I've also tested it with "gutsy". You just have to get the "7.10" alternate cd "iso" file and the matching set of installer files from:

In fact, when new versions arrive, all you should have to do is change vmlinuz/initrd, copy in the new alternate "iso" file and your ready to go.

Configuring ''grub''

Next step is to set up the grub configuration file (menu.lst) located in /media/Bootie/boot/grub. MAKE SURE THAT YOU EDIT THE ONE ON THE CF/USB DISK!! Using gedit (or your favorite editor), open the file:

  • root@hairpin:/media/Bootie# gedit ./boot/grub/menu.lst &

Change the file so that it contains the following lines. Note that all the disk references are now set to the first partition on hd0 - this will in fact be the drive designation of the CF/USB when you boot from it:


default         0
timeout         3

title   Install in Text Mode
kernel  (hd0,0)/boot/vmlinuz fb=false video=vga16:off root=/dev/ram0 ramdisk_size=13000 quiet
initrd  (hd0,0)/boot/initrd.gz

title   Install in Expert Text Mode
kernel  (hd0,0)/boot/vmlinuz fb=false video=vga16:off priority=low root=/dev/ram0 ramdisk_size=13000 quiet
initrd  (hd0,0)/boot/initrd.gz

title   Rescue a broken system
kernel  (hd0,0)/boot/vmlinuz rescue/enable=true fb=false video=vga16:off root=/dev/ram0 ramdisk_size=13000 quiet
initrd  (hd0,0)/boot/initrd.gz


Note that your browser may break up the kernel line. When you copy and paste them, make sure that the three directives (title, kernel and initrd) each land on one line (i.e., no embedded <cr>'s). The items in this menu are intended to allow you to choose between several options that are provided on the alternate CD. I have only tested the default installer ("Install in Text Mode") and cannot vouch for the functionality of either of the remaining 2 options.

When done, close the file and unmount the CF/USB disk:

  • root@hairpin:/media/Bootie# cd ../..
    root@hairpin:/#umount /media/Bootie

Note that it's important that you unmount the CF/USB disk before running grub. You may get away with it if you don't, but the documentation warns that this may cause "inconsistencies in the disk I/O" (whatever that means).

Installing the bootloader

Now run grub to install the boot loader on the CF/USB disk:

  • root@hairpin:#grub
    Probing devices to guess BIOS drives. This may take a long time.

You probably won't see the previous line (at least for long enough to read it). While in grub, the terminal screen is replaced with the grub shell (below), hiding the initial response of grub. You will see it later when you exit grub. I've tried to illustrate the major points with some of the most useful grub features. Comments appear as appropriate. What follows then, is the grub dialog.


Grub starts

               [ Minimal BASH-like line editing is supported.   For
                 the   first   word,  TAB  lists  possible  command
                 completions.  Anywhere else TAB lists the possible
                 completions of a device/filename.  ESC at any time
                 exits. ]

The first thing to do is to absolutely confirm the correct drive designation. This is where the "id.txt" file comes in handy - providing of course that you don't foolishly place an identical file in the root of your system drive! We'll use grub's find command to locate the file:

  • grub> find /id.txt
     (hd1,0)

The response from grub - "(hd1,0)" - has just told us that the CF/USB disk partition that contains the installer is "(hd1,0)". It has also illustrated an important bit of syntax: drives are designated in parentheses as "(hdx,y)", where x is the drive number and y is the partition number. We'll use this info in the next 2 steps.

Set The grub root.

Next, we must set the grub "root". This is so that grub can make adjustments to the loader that it installs so that it will boot properly when the drive letters are later changed by the BIOS during the actual boot - remember, it's "(hd1,0)" now, but will be "(hd0,0)" at boot time. I've entered only root (hd<tab> to show how grub will list all available devices using <tab> completion. We don't want hd0! This is the main, 100GB drive on which Ubuntu is already installed. If we make this the root, then when our CF/USB drive boots it will try to find files on places that it shouldn't:

  • grub> root (hd
     Possible disks are:  hd0 hd1

Next, I add "1,<tab>" to the command to see what partitions I have available:

  • grub> root (hd1,
     Possible partitions are:
       Partition num: 0,  Filesystem type is ext2fs, partition type 0x83
       Partition num: 1,  Filesystem type is fat, partition type 0x6

I'm working with my USB device here, so there are 2 partitions. I want partition 0 (note that sdb1 is partition 0!). In the following command, I choose partition 0 by completing the command with "0)"<cr>. You may have only one partition, in which case grub will complete the line for you if you type a <tab>. I choose partition 0 by completing the command with "0)"<cr>:

  • grub> root (hd1,0)

Setup and Install the grub Bootstrap Code.

Since I know that the grub loader files are on hd1,0 (I copied them onto the CF after all), next I'll let grub set itself up on hd1 (the CF/USB drive). We don't specify a partition here since we are setting up the boot loader in the Master Boot Record (MBR) which resides outside any partition:

  • grub> setup (hd1)
     Checking if "/boot/grub/stage1" exists... yes
     Checking if "/boot/grub/stage2" exists... yes
     Checking if "/boot/grub/e2fs_stage1_5" exists... yes
     Running "embed /boot/grub/e2fs_stage1_5 (hd1)"...  17 sectors are embedded.
    succeeded
     Running "install /boot/grub/stage1 (hd1) (hd1)1+17 p (hd1,0)/boot/grub/stage2 /boot/grub/menu.lst"... succeeded
    Done.

Notice what's happened. Grub has detected the filesystem type and patched the stage 2 loader with the stage 1.5 fragment. This teaches stage2 how to deal with the ext2 filesystem (that's the "embed" command). It has then copied the stage1 file to the MBR of the CF/USB disk and followed it with the (now patched) stage2. As far as grub is concerned, it's done. So now we just quit grub and return to the root command prompt.

  • grub> quit


Grub ends

Booting the ''CF/USB'' Drive

At this point, you should be able to boot from the newly created media. The trick now is how to instruct your BIOS to boot from the device. On the Advantech PCM9381, I need to select HDD-1 for the CF device and USB-HDD for the USB memory stick. Unfortunately, every BIOS is different. Sometimes, it is clear from the wording of the choices, other times it is necessary to try virtually every combination until you find one that works. Once you have gotten the BIOS set up to boot from the CF/USB disk, grub will start and present you with the menu that you installed in the "menu.lst" file. I choose the "Text Mode" installer. The rest is standard stuff.

Installation/FromCForUSBStick (last edited 2011-06-02 09:45:37 by skovprodukter)