Diff for "Partitioning/Home/Moving"


Differences between revisions 28 and 75 (spanning 47 versions)
Revision 28 as of 2010-04-13 15:13:06
Size: 7492
Editor: 78-105-201-166
Comment:
Revision 75 as of 2015-11-13 13:16:06
Size: 12622
Editor: 80
Comment: Towards the end, I swapped the order of "Preparing fstab for the switch" and "Moving /home into /old_home". Once you rename home, you will no longer have access to gedit.
Deletions are marked like this. Additions are marked like this.
Line 3: Line 3:
=== The Advantages of This Guide ===
This guide was written simply because there were flaws in the other guides and it was worth recording these notes for future reference. Community Documentation is an excellent way to help others at the same time. This guide:
 * Aims to keep the system usable and the data all safe.
  * If the system unexpectedly loses power or goes into hibernate mode or anything at any point through this guide then it should still boot-up & work just fine. That's the main aim anyway.
 * Prepares Partitions and fstab first then only tweaks fstab at the end
  * People test and gain familiarity with the tools before the final crunch-test, another advantage of this guide. Some other guides use mount commands, then move the /home & then set-up fstab. However, if you mess up your fstab at that crunch-point then when you reboot your system will lose track of your /home files! Setting up fstab first and using "sudo mount -a" to mount the partitions checks the modified fstab works at a safe point in the process.
 * Use rsync to move the files
  * [[rsync | Rsync]] is designed for backups of /home, and much more. It is able to maintain other characteristics of the file, like permissions, ownership, and timestamps. There was a lot of debate about which command to copy your files to the new /home partition which stemmed from a time when cp was not able to do it properly (apparently it skipped files?). Cp was not designed to be a powerful backup tool, rsync was. The fix, then, was to use a combination of find and cpio ([[http://www.us.debian.org/doc/manuals/reference/ch-tips.en.html#s-archiving|See section 8.3.5]]).

= The Guide =
Setting up /home on a separate partition is beneficial because your settings, files, and desktop will be maintained if you upgrade, (re)install Ubuntu or another distro. This works because /home has a sub-folder for each user's settings and files which contain all the data & settings of that user. Also, fresh installs for linux typically like to wipe whatever partition they are being installed to so either the data & settings need to be backed-up elsewhere or else avoid the fuss each time by having /home on a different partition.
=== Overview ===

This guide offers detailed instructions for migrating your home directory into its own dedicated partition. Setting up /home on a separate partition is beneficial because your settings, files, and desktop will be maintained if you upgrade, (re)install Ubuntu or another distro. This works because /home has a subdirectory for each user's settings and files which contain all the data & settings of that user. Telling Ubuntu to use an existing home partition can be done by selecting "Manual Partitioning" during the installation of Ubuntu and specifying that you want your home partitions mount point to be /home, '''ensure you mark your /home partition not be formatted in the process'''. You should also make sure the usernames you enter for accounts during installation match usernames that existed in a previous installation.

This guide will follow these 8 basic steps:

 1. Set-up your new partition
 2. Find the uuid (=address) of the new partition
 3. Backup and edit your fstab to mount the new partition as /media/home (just for the time being) and reboot.
 4. Use rsync to migrate all data from /home into /media/home
 5. Check copying worked!
 6. Move /home to /old_home to avoid confusion later!
 7. Edit fstab again so the new partition mounts as /home instead of as /media/home
 8. Reboot or remount all. Check system seems to be working well
 9. Delete the /old_home after a while

The guide is written in such a way so that at any point in time if there is a system failure, power outage or random restart that it will not have a negative impact on the system and ''SHOULD'' safeguard against the possibility of the user accidentally deleting their home directory in the process.

= Creating a new partition =
Setting up /home on a separate partition is beneficial because your settings, files, and desktop will be maintained if you upgrade, (re)install Ubuntu or another distro. This works because /home has a subdirectory for each user's settings and files which contain all the data & settings of that user. Also, fresh installs for linux typically like to wipe whatever partition they are being installed to so either the data & settings need to be backed-up elsewhere or else avoid the fuss each time by having /home on a different partition.
Line 16: Line 26:
This is beyond the scope of this page. [[HowtoPartition|Try here if you need help]]. Memorize or write down the location of the partition, something like /sda3. Its been suggested to use either ext2, ext3 or ext4 rather than vfat or ntfs. Using vfat is not supported and may fail, since vfat does not support permissions. Ntfs has an advantage of being journalised, like ext3 & ext4 but it can also be read easily by Windows. It is a lot less stable than ext3 or ext4 so it is really a question of stability compared with easy access for Windows. If you don't dual-boot with Windows or have plenty of hard-drive space then staying with very much safer ext3 or 4 is the obvious choice. This is beyond the scope of this page. [[HowtoPartition|Try here if you need help]]. Memorize or write down the location of the partition, something like /sda3. When you do create a new partition it is highly suggested that you create an ext3 or ext4 partition to house your new home directory.
Line 20: Line 30:
The uuid reference to all partitions get to a [[https://help.ubuntu.com/community/UsingTheTerminal#Starting%20a%20Terminal | command-line]] to try this The uuid (Universally Unique Identifier) reference for all partitions can be found by opening a [[https://help.ubuntu.com/community/UsingTheTerminal#Starting%20a%20Terminal | command-line]] to type the following:
Line 24: Line 34:
Now you just need to note down (copy&paste into a text-file) the uuid of the partition that you have set-up ready to be the new /home partition.  Some older releases of Ubuntu might not have the "blkid" command so this could be used instead
===
Alternative Method ===
Fo
r some older releases of Ubuntu the "blkid" command might not work so this could be used instead
Line 32: Line 44:
Now you just need to take note (copy&paste into a text-file) the uuid of the partition that you have set-up ready to be the new /home partition.
Line 36: Line 49:
These commands should;
1. Create a backup of fstab. Replace "DateToday" with today's date so you can find the right back-up easily if you do need to use it later.
2. Open the standard text-editor to edit fstab. Note that Kubuntu uses "kate" & Xubuntu uses "mousepad" instead of "gedit" you could use a different text-editor if you prefer.
{{{
sudo cp /etc/fstab /etc/fstab.DateToday
gksu gedit /etc/fstab
}}}
and add these lines into it
{{{
Your fstab is a file used to tell Ubuntu what partitions to mount at boot. The following commands will duplicate your current fstab, append the year-month-day to the end of the file name, compare the two files and open the original for editing.

1. Duplicate your fstab file:
{{{
sudo cp /etc/fstab /etc/fstab.$(date +%Y-%m-%d)
}}}

2. Compare the two files to confirm the backup matches the original:
{{{
cmp /etc/fstab /etc/fstab.$(date +%Y-%m-%d)
}}}

3. Open the original fstab in a text editor:
{{{
sudo gedit /etc/fstab
}}} and add these lines into it {{{
Line 46: Line 66:
UUID=???????? /media/home ext3 nodev,nosuid 0 2
}}}
Replace???????? with the UUID number of the intended /home partition.

The Fstab location should be a temporary one - we are to mount the temporary location and copy the existing home to it. (/media/home is assumed in the rest of the guide). If /media/home does not exist yet, then you should create it with the command:
UUID=???????? /media/home ext3 defaults 0 2
}}} and replace the "????????" with the UUID number of the intended /home partition.

NOTE: In the above example, the specified partition in the new text is an ext3, but if yours is an ext4 partition, you should change the part above that says "ext3" to say "ext4", in addition to replacing the ???'s with the correct UUID. Also note that if you are using Kubuntu, Xubuntu or Lubuntu you may need to replace "gedit" with "kate", "mousepad" or "leafpad", respectively. They are text editors included with those distributions.


4. Save and Close the fstab file, then type the following command:
Line 55: Line 77:
Now, mount the partition with: This command will create a new directory, later used for temporarily mounting the new partition. At the end of the procedure this directory can be removed.

Now you can restart your machine or instead of rebooting you might prefer to just re-load the updated fstab
Line 60: Line 84:
Check that you can copy files over, at least as root (ie with sudo). Either should have mounted the new partition as /media/home. We will edit the fstab again later so this arrangement of the partition is only temporary.
Line 64: Line 88:
{{{
sudo rsync -axS --exclude='/*/.gvfs' /home/. /media/home/.
}}}
The --exclude='/*/.gvfs' prevents rsync from complaining about not being able to copy .gvfs, but I believe it optional. Even if rsync complains, it will copy everything else anyway. ([[http://ubuntuforums.org/showthread.php?t=791693|See here for discussion on this]])



== Make the Switch ==
Here is the point of no return. If the previous two steps worked, you should have no troubles. The data in /home now exists in 2 places and we need fstab to point to the new one. so again on a command-line
{{{
gksu gedit /etc/fstab
}}}
and now edit the lines you added earlier to delete the /media part. This should make /media/home into /home as follows

Next we will copy all files, directories and sub-directories from your current /home directory into the new partition.
If you do not have an encrypted home file system, just do the following:

{{{
sudo rsync -aXS --exclude='/*/.gvfs' /home/. /media/home/.
}}}
I prefer adding the "--progress" tag just before the "--excludes" one - otherwise there is no indication of anything happening. The "--progress" tag reports on each file individually so you see tons of unfamiliar stuff scrolling by very fast. Rsync can be interrupted as many times as you like and it checks to see how much has already been done when you start it up again. So, this copying stage can be broken down into many sessions. After it has completed once it's wise to run it a couple more times just to make sure it includes everything you may have added since first starting the first copying/syncing session - even if you've done the whole thing all in just one session.

The --exclude='/*/.gvfs' prevents rsync from complaining about not being able to copy .gvfs, but I believe it is optional. Even if rsync complains, it will copy everything else anyway. ([[http://ubuntuforums.org/showthread.php?t=791693|See here for discussion on this]])

=== Encrypted file systems ===
If you have an encrypted home file system, then the above will just leave you with an unencrypted copy of your files,
which is probably not what you want. You could re-encrypt them after copying, or copy them in their encrypted form.
Here is one way to do that.

First, you'll need to shut down, and reboot from a LiveCD or USB stick. Then you'll need to mount your root partition
and new home partition. (You can do this by selecting those devices in the file viewer). They will be mounted under
/media/ubuntu - so for example, if you named your root partition linux-root, then your old home directory will be found at
/media/ubuntu/linux-root/home. And if you named your new home partition linux-home, then this will be found at
/media/ubuntu/linux-home. So, now you can copy your encrypted home files
(here assuming your partitions are named linux-root and linux-home):

{{{
sudo rsync -aXS /media/ubuntu/linux-root/home/. /media/ubuntu/linux-home/.
}}}
There is no point trying to exclude any files with specific names, because the names of the files are encrypted too!

Leave your machine running from the LiveCD or USB for the moment.


== Check Copying Worked ==
You should now have two duplicate copies of all the data within your home directory; the original being located in /home and the new duplicate located in /media/home. You should confirm all files and directories copied over successfully. One way to do this (for an unencrypted file system) is by using the diff command:

{{{
sudo diff -r /home /media/home -x ".gvfs/*"
}}}

If you are doing this from a LiveCd or to an existing partition that already has stuff on it you may find differences but hopefully it should be obvious which diffs you can ignore.

You can also expect to see some errors about files not found. These are due to symbolic links that point to places that don't presently exist (but will do after you have rebooted). You can ignore these - but check out anything else.

=== Encrypted file systems ===
If you have an encrypted file system, the command will look more like this.
{{{
sudo diff -r /media/ubuntu/linux-root/home /media/ubuntu/linux-home
}}}

Now you can shut-down, remove the LiveCD / USB stick, and reboot as normal.


== Preparing fstab for the switch ==
We now need to modify the fstab again to point to the new partition and mount it as /home. So again on a command-line
{{{
sudo gedit /etc/fstab
}}}
and now edit the lines you added earlier, changing the "/media/home" part to simply say "/home" so that it looks like this:
Line 79: Line 147:
UUID=???????? /home ext3 nodev,nosuid 0 2
}}}


Rename your /home directory:
{{{
sudo mv /home /old_home
}}}

Re-create a /home directory with (this is because the next steps need the location to already exist):
{{{
sudo mkdir /home
}}}

~-''It seems that this guide fails at this point (under 9.04) because sudo fails as it can not find the config files it needs to check the users sudo permissions. This is because /home no longer exists. To fix this I needed to start a shell as root (su command and supply the root password) and create the new /home. - Roger Morton ''-~


Edit fstab so that your new home partition actually points to /home (ie. by changing /media/home to /home)

~-''gksudo would not launch gedit (for the same reason sudo failed above I suspect) so I was forced to use vi to edit my fstab. And I hate vi. - Roger Morton ''-~


and finally, remount the partition with:
{{{
UUID=???????? /home ext3 defaults 0 2
}}}

Then, press Save, close the file but don't reboot just yet.


== Moving /home into /old_home ==

Backing up your old home, just in case things have not gone completely smoothly, is best done right now. Here is how:

As long as you have not rebooted yet, you will still see 2 copies of your /home directory; the new one on the new partition (currently mounted as /media/home) and the old one still in the same partition it was always in (currently mounted as /home). We need to move the contents of the old home directory out of the way and create an empty "place-holder" directory to act as a "mount point" for our new partition.

Type the following string of commands in to do all this at once:
{{{
cd / && sudo mv /home /old_home && sudo mkdir /home
}}}

By default, when you open a terminal window it places you within your home directory. Typing cd / takes us to the root directory and out of home so we can then use the sudo mv command to essentially rename /home into /old_home, and finally create a new, empty /home placeholder.



== Reboot or Remount all ==
With;
 1. your fstab now edited to mount your new partition to our /home place-holder and
 2. the original /home now called /old_home,

it should be a good time to reboot your computer to check the whole thing really did work. Your new partition should mount as /home and everything should look exactly the same as it did before you started.

Btw, geeks might prefer to avoid rebooting by just re-loading the updated fstab
{{{
Line 105: Line 179:

~-''I recall that this failed too. I think because home is already mounted - even though we have renamed it. umount /home failed because /home was in use by the GUI. In the end I had to reboot and bring up the recovery console as root and fix the mess. So bottom line - I don't think this is such a good guide. Needs further work and testing. - Roger Morton''-~


Reboot to ensure that currently running programs use the new /home

== After a reboot ==
If everything is working, great, you can delete /old_home. If things aren't working, you can undo by moving your /old_home back to /home, and delete the line you added to fstab.
There is no need to reboot - unless you have an encrypted file system.


=== Troubleshooting ===
If you receive an error like 'The volume may already be mounted', use the following command to unmount the drive first before re-doing the last step again. (note the "n" should be missing from the command, making it "umount")
{{{
sudo umount /media/home/
}}}

Then try mounting again
{{{
sudo mount -a
}}}


== Deleting the old Home ==

You can keep using the system as it is for ages before doing this, unless you are desperately short of space on the / partition. It's probably best to leave this step until a long time after you have been using the system happily. When you do eventually feel safe enough to delete the old home then you can try;
{{{
cd /
sudo rm -rI /old_home
}}}
Be careful with the above command because mis-typing it could result in the deletion of other files and directories!
Line 121: Line 210:

== Different filesystems on the same disk ==

If you're moving from Windows and your new home partition is going to be an old ntfs partition (the D: disk) while you convert the C: disk to a journaling partition where you install Linux, this won't work, there will be a huge load on the processor. You should convert the two partitions to ext3 or ext4 or keep both partitions as ntfs (I haven't checked this last option). But working with two different filesystems on the same drive simultaneously doesn't seem to be a good option.

Overview

This guide offers detailed instructions for migrating your home directory into its own dedicated partition. Setting up /home on a separate partition is beneficial because your settings, files, and desktop will be maintained if you upgrade, (re)install Ubuntu or another distro. This works because /home has a subdirectory for each user's settings and files which contain all the data & settings of that user. Telling Ubuntu to use an existing home partition can be done by selecting "Manual Partitioning" during the installation of Ubuntu and specifying that you want your home partitions mount point to be /home, ensure you mark your /home partition not be formatted in the process. You should also make sure the usernames you enter for accounts during installation match usernames that existed in a previous installation.

This guide will follow these 8 basic steps:

  1. Set-up your new partition
  2. Find the uuid (=address) of the new partition
  3. Backup and edit your fstab to mount the new partition as /media/home (just for the time being) and reboot.
  4. Use rsync to migrate all data from /home into /media/home
  5. Check copying worked!
  6. Move /home to /old_home to avoid confusion later!
  7. Edit fstab again so the new partition mounts as /home instead of as /media/home
  8. Reboot or remount all. Check system seems to be working well
  9. Delete the /old_home after a while

The guide is written in such a way so that at any point in time if there is a system failure, power outage or random restart that it will not have a negative impact on the system and SHOULD safeguard against the possibility of the user accidentally deleting their home directory in the process.

Creating a new partition

Setting up /home on a separate partition is beneficial because your settings, files, and desktop will be maintained if you upgrade, (re)install Ubuntu or another distro. This works because /home has a subdirectory for each user's settings and files which contain all the data & settings of that user. Also, fresh installs for linux typically like to wipe whatever partition they are being installed to so either the data & settings need to be backed-up elsewhere or else avoid the fuss each time by having /home on a different partition.

Setup Partitions

This is beyond the scope of this page. Try here if you need help. Memorize or write down the location of the partition, something like /sda3. When you do create a new partition it is highly suggested that you create an ext3 or ext4 partition to house your new home directory.

Find the uuid of the Partition

The uuid (Universally Unique Identifier) reference for all partitions can be found by opening a command-line to type the following:

sudo blkid

Alternative Method

For some older releases of Ubuntu the "blkid" command might not work so this could be used instead

sudo vol_id -u <partition>

for example

sudo vol_id -u /dev/sda3

Now you just need to take note (copy&paste into a text-file) the uuid of the partition that you have set-up ready to be the new /home partition.

Setup Fstab

Your fstab is a file used to tell Ubuntu what partitions to mount at boot. The following commands will duplicate your current fstab, append the year-month-day to the end of the file name, compare the two files and open the original for editing.

1. Duplicate your fstab file:

sudo cp /etc/fstab /etc/fstab.$(date +%Y-%m-%d)

2. Compare the two files to confirm the backup matches the original:

cmp /etc/fstab /etc/fstab.$(date +%Y-%m-%d)

3. Open the original fstab in a text editor:

sudo gedit /etc/fstab 

and add these lines into it

# (identifier)  (location, eg sda5)   (format, eg ext3 or ext4)      (some settings) 
UUID=????????   /media/home    ext3          defaults       0       2 

and replace the "????????" with the UUID number of the intended /home partition.

NOTE: In the above example, the specified partition in the new text is an ext3, but if yours is an ext4 partition, you should change the part above that says "ext3" to say "ext4", in addition to replacing the ???'s with the correct UUID. Also note that if you are using Kubuntu, Xubuntu or Lubuntu you may need to replace "gedit" with "kate", "mousepad" or "leafpad", respectively. They are text editors included with those distributions.

4. Save and Close the fstab file, then type the following command:

sudo mkdir /media/home

This command will create a new directory, later used for temporarily mounting the new partition. At the end of the procedure this directory can be removed.

Now you can restart your machine or instead of rebooting you might prefer to just re-load the updated fstab

sudo mount -a

Either should have mounted the new partition as /media/home. We will edit the fstab again later so this arrangement of the partition is only temporary.

Copy /home to the New Partition

Next we will copy all files, directories and sub-directories from your current /home directory into the new partition. If you do not have an encrypted home file system, just do the following:

sudo rsync -aXS --exclude='/*/.gvfs' /home/. /media/home/.

I prefer adding the "--progress" tag just before the "--excludes" one - otherwise there is no indication of anything happening. The "--progress" tag reports on each file individually so you see tons of unfamiliar stuff scrolling by very fast. Rsync can be interrupted as many times as you like and it checks to see how much has already been done when you start it up again. So, this copying stage can be broken down into many sessions. After it has completed once it's wise to run it a couple more times just to make sure it includes everything you may have added since first starting the first copying/syncing session - even if you've done the whole thing all in just one session.

The --exclude='/*/.gvfs' prevents rsync from complaining about not being able to copy .gvfs, but I believe it is optional. Even if rsync complains, it will copy everything else anyway. (See here for discussion on this)

Encrypted file systems

If you have an encrypted home file system, then the above will just leave you with an unencrypted copy of your files, which is probably not what you want. You could re-encrypt them after copying, or copy them in their encrypted form. Here is one way to do that.

First, you'll need to shut down, and reboot from a LiveCD or USB stick. Then you'll need to mount your root partition and new home partition. (You can do this by selecting those devices in the file viewer). They will be mounted under /media/ubuntu - so for example, if you named your root partition linux-root, then your old home directory will be found at /media/ubuntu/linux-root/home. And if you named your new home partition linux-home, then this will be found at /media/ubuntu/linux-home. So, now you can copy your encrypted home files (here assuming your partitions are named linux-root and linux-home):

sudo rsync -aXS /media/ubuntu/linux-root/home/. /media/ubuntu/linux-home/.

There is no point trying to exclude any files with specific names, because the names of the files are encrypted too!

Leave your machine running from the LiveCD or USB for the moment.

Check Copying Worked

You should now have two duplicate copies of all the data within your home directory; the original being located in /home and the new duplicate located in /media/home. You should confirm all files and directories copied over successfully. One way to do this (for an unencrypted file system) is by using the diff command:

sudo diff -r /home /media/home -x ".gvfs/*"

If you are doing this from a LiveCd or to an existing partition that already has stuff on it you may find differences but hopefully it should be obvious which diffs you can ignore.

You can also expect to see some errors about files not found. These are due to symbolic links that point to places that don't presently exist (but will do after you have rebooted). You can ignore these - but check out anything else.

Encrypted file systems

If you have an encrypted file system, the command will look more like this.

sudo diff -r /media/ubuntu/linux-root/home /media/ubuntu/linux-home

Now you can shut-down, remove the LiveCD / USB stick, and reboot as normal.

Preparing fstab for the switch

We now need to modify the fstab again to point to the new partition and mount it as /home. So again on a command-line

sudo gedit /etc/fstab

and now edit the lines you added earlier, changing the "/media/home" part to simply say "/home" so that it looks like this:

# (identifier)  (location, eg sda5)   (format, eg ext3 or ext4)      (some settings) 
UUID=????????   /home    ext3          defaults       0       2

Then, press Save, close the file but don't reboot just yet.

Moving /home into /old_home

Backing up your old home, just in case things have not gone completely smoothly, is best done right now. Here is how:

As long as you have not rebooted yet, you will still see 2 copies of your /home directory; the new one on the new partition (currently mounted as /media/home) and the old one still in the same partition it was always in (currently mounted as /home). We need to move the contents of the old home directory out of the way and create an empty "place-holder" directory to act as a "mount point" for our new partition.

Type the following string of commands in to do all this at once:

cd / && sudo mv /home /old_home && sudo mkdir /home

By default, when you open a terminal window it places you within your home directory. Typing cd / takes us to the root directory and out of home so we can then use the sudo mv command to essentially rename /home into /old_home, and finally create a new, empty /home placeholder.

Reboot or Remount all

With;

  1. your fstab now edited to mount your new partition to our /home place-holder and
  2. the original /home now called /old_home,

it should be a good time to reboot your computer to check the whole thing really did work. Your new partition should mount as /home and everything should look exactly the same as it did before you started.

Btw, geeks might prefer to avoid rebooting by just re-loading the updated fstab

sudo mount -a

There is no need to reboot - unless you have an encrypted file system.

Troubleshooting

If you receive an error like 'The volume may already be mounted', use the following command to unmount the drive first before re-doing the last step again. (note the "n" should be missing from the command, making it "umount")

sudo umount /media/home/

Then try mounting again

sudo mount -a

Deleting the old Home

You can keep using the system as it is for ages before doing this, unless you are desperately short of space on the / partition. It's probably best to leave this step until a long time after you have been using the system happily. When you do eventually feel safe enough to delete the old home then you can try;

cd /
sudo rm -rI /old_home

Be careful with the above command because mis-typing it could result in the deletion of other files and directories!

Technical Notes and Resources

Rsync was chosen over cp and find|cpio because it seemed to maintain permissions.

http://ubuntu.wordpress.com/2006/01/29/move-home-to-its-own-partition/

http://ubuntuforums.org/showthread.php?t=46866

Different filesystems on the same disk

If you're moving from Windows and your new home partition is going to be an old ntfs partition (the D: disk) while you convert the C: disk to a journaling partition where you install Linux, this won't work, there will be a huge load on the processor. You should convert the two partitions to ext3 or ext4 or keep both partitions as ntfs (I haven't checked this last option). But working with two different filesystems on the same drive simultaneously doesn't seem to be a good option.

Partitioning/Home/Moving (last edited 2015-11-13 13:16:06 by 80)