How to install upgrade an existing Ubuntu install onto a Linux Software RAID system
Note: If you are looking for how to initially install Ubuntu onto software raid please visit https://help.ubuntu.com/community/Installation/SoftwareRAID
Introduction
RAID is a method of using multiple disks together in different configurations to improve or enhance the disk capabilities of a system running RAID. Raid is available as a hardware solution, provided by expensive disk controller hardware or a software solution where multiple disks are managed by a software process to control the RAID functionality. This document will cover only the software RAID functionality.
This document will cover how to implement a software RAID configuration onto an existing running Ubuntu Linux computer, it will be applicable to any Official Ubuntu variant, 32 bit and 64 bit, Desktop or Server release.
Requirements
- Enough physical disk devices and space on those devices to meet your RAID requirements
- The ability to reboot your machine
- A supported Ubuntu Live CD (this is optional as this can be done from within the running operating system, but risk can be reduced using a currently supported Ubuntu Live CD).
- A basic understanding of the RAID technology you want to use to create your solution.
- A basic awareness of Linux command line interaction.
Limitations
- Only RAID 1 is supported for the /boot file system, if you want to use anything over than RAID 1 for your root file system make sure you use a seperate /boot partition that does either not use RAID, or only uses RAID 1.
Installing
This section will cover the multiple RAID conversion processes that are commonly requested. The example platform used in all examples is Ubuntu Server 64bit 8.10 running in a KVM Virtual machine with 3 4Gigabyte Disks. The initial starting layout for all the examples will be a single disk installation partitioned in the following layout.
This is used purely for example purposes and is not a recommended or approved file system layout. If you are not confident setting up your own file system layout then this guide should not be used until you are confident with basic disk layout.
Converting to a RAID 1 Mirrored System
One of the most common RAID configurations is a RAID 1 mirrored system
Configuring the RAID
- Once you have complete your partitioning in the main "Partition Disks" page select "Configure Software RAID"
- Select "Yes"
- Select "Create new MD drive"
- Select RAID1, or type of RAID you want (RAID0 RAID1 RAID5)
- Number of devices 2 or the amount of hard drives you have
- Number of spare devices 0
- select which partitions to use. Generally they will be sda1 and sdb1 or hda1 or hdb1. Generally the numbers will match and the different letters are for different hard drives.
- At this point the installation may become unresponsive this is the hard drives already syncing. Repeat steps 3 to 7 with each pair of partitions you have created.
- Once done, select finish.
Formatting
You now have a list of your hard drives and your RAID drives. We will now format and set the mount point for other RAID drives. Treat the RAID drive as a local hard drive and format and mount accordingly
- Select Partition.
- Go to Use as Select Ext 3 for your normal partitions or swap area for your swap partition
- If you select Ext 3 then select your mount point if you only have one partition for ext 3 select /
- Repeat for each RAID partition.
Select "Finish partitioning and write changes to disk"
From this point on, your hard drive lights will probably be on continuously. This indicates the array syncing. The system can be used normally and even rebooted while the array syncs.
Boot Loader
Installation continues as normal until you have to install a bootloader. GRUB installation may appear to complete successfully, but it will actually work. The key is that GRUB can't be installed on the RAID device. However, the individual raw partitions that make up the RAID device look just like ordinary non-RAIDed partitions to GRUB. So you need to unmount the RAID device and install grub manually. There are a few ways to go about this, but probably the easiest is to let installation finish and allow the system to begin rebooting. You'll end up at a
GRUB>
prompt. Now
GRUB> find /boot/grub/menu.lst
or if you have a separate /boot partition,
GRUB> find /grub/menu.lst
GRUB should respond with something like
(hd0,0) (hd1,0)
The above says your menu.lst file was found in the first partition of each of your first two disks.
Then, for each (hdX,Y) line you get in response, use this pattern
GRUB> root (hdX,Y) GRUB> setup(hdX)
When done, hit CTRL-ALT-DEL to reboot the machine again, and everything should work.
Note: I'm not 100% sure the above procedure works, because in my case there was probably already a grub hanging around in the MBR of (hd0), and I don't know whether the regular installer procedure will put one there when /boot is on RAID. Without GRUB in the MBR, you wouldn't even get to a GRUB> prompt at boot. If that fails, please see my alternate method at http://techarcana.net/hydra/os-installation/#boot-loader, and please let me know whether this works for you or not.
If for some reason you are able to boot without being dropped into a GRUB> prompt, you'll still want to install GRUB in the MBR (usually) of each drive in the RAID1 array that holds /boot (rather than just the first drive), so that if one drive goes down you can still boot. So even if things seem to be working, it's a good idea to hit escape at boot to bring up the GRUB menu, drop into a GRUB> prompt, and do the above exercise.
Updating startup script
Every time the computer boots, it scans the available hard-drives to try and identify any RAID array. Most of the time this is easy and takes place instantaneously. If one of the drives is unusable, however, then the computer needs to operate the remaining drive in 'degraded' mode. If the bad drive failed during regular operation, then the computer will have already removed it from the configuration of the array. If the drive had previously been working, however, and failed spontaneously during power-up or boot-up, then the computer needs to figure it out on-the-fly.
The script which tries to detect a failed drive is called 'initramfs' and, as of Ubuntu 8.04, the default code in this script completely fails. The procedure below adds an additional step so that it will succeed.
1. Update the 'initramfs' boot script,
> gksudo gedit /usr/share/initramfs-tools/scripts/local
2. Find the comment,
# We've given up, but we'll let the user fix matters if they can".
3. Just *before* this comment, add the following:
# The following code was added to allow degraded RAID arrays to start
if [ ! -e "${ROOT}" ] || ! /lib/udev/vol_id "${ROOT}" >/dev/null 2>&1; then
# Try mdadm and allow degraded arrays to start in case a drive has failed
log_begin_msg "Attempting to start RAID arrays and allow degraded arrays"
/sbin/mdadm --assemble --scan
log_end_msg
# If you use logical volume on raid partition, is better wait some seconds or the boot will fail!
sleep 10
fi4. Save the change and exit the editor.
5. Finally, update the boot image to use the updated script,
> sudo update-initramfs -u
Troubleshooting
Swap space doesn't come up, error message in dmesg
Provided the RAID is working fine this can be fixed with
> sudo update-initramfs -k all -u
Resources
https://wiki.ubuntu.com/HotplugRaid Keeping your data synced and mirrored on external drives.
Using mdadm
Checking the status of your RAID
Two useful commands to check the status are:
cat /proc/mdstat
This will show output something similar to
Personalities : [raid1] [raid6] [raid5] [raid4]
md5 : active raid1 sda7[0] sdb7[1]
62685504 blocks [2/2] [UU]
md0 : active raid1 sda1[0] sdb1[1]
256896 blocks [2/2] [UU]
md6 : active raid5 sdc1[0] sde1[2] sdd1[1]
976767872 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]From this information you can see that the available personalities on this machine are "raid1, raid6, raid4, and raid5" which means this machine is setup to use raid devices configured in a raid1, raid6, raid4 and raid5 configuration.
You can also see in the three example meta devices that there are two raid 1 mirrored meta devices. These are md0 and md5. You can see that md5 is a raid1 array and made up of disk /dev/sda partition 7, and /dev/sdb partition 7, containing 62685504 blocks, with 2 out of 2 disks available and both in sync.
The same can be said of md0 only it is smaller (you can see from the blocks parameter) and is made up of /dev/sda1 and /dev/sdb1.
md6 is different in that we can see it is a raid 5 array, striped across 3 disks. These are /dev/sdc1, /dev/sde1 and /dev/sdd1, with a 64k "chunk" size which is basically a "write" size. Algorithm 2 shows it is a write algorithm patern 2 which is "left disk to right disk" writing across the array. You can see that all 3 disks are present and in sync.
sudo mdadm --query --detail /dev/md*
( where * is the partition number)