Diff for "Partitioning/Home/Moving"


Differences between revisions 33 and 73 (spanning 40 versions)
Revision 33 as of 2010-08-04 14:08:00
Size: 7263
Editor: cpc3-cmbg10-0-0-cust345
Comment: someone added some clever bash coding so i altered instructions to match
Revision 73 as of 2015-11-02 18:38:39
Size: 12305
Editor: host-92-11-32-239
Comment: Added the extra steps to make the list in the intro match the Table of Contents
Deletions are marked like this. Additions are marked like this.
Line 3: Line 3:
=== The Advantages of This Guide ===
This guide was written simply because there were flaws in the other guides and it was worth recording these notes for future reference. Community Documentation is an excellent way to help others at the same time. This guide:
 * Aims to keep the system usable and the data all safe.
  * If the system unexpectedly loses power or goes into hibernate mode or anything at any point through this guide then it should still boot-up & work just fine. That's the main aim anyway.
 * Prepares Partitions and fstab first then only tweaks fstab at the end
  * People test and gain familiarity with the tools before the final crunch-test, another advantage of this guide. Some other guides use mount commands, then move the /home & then set-up fstab. However, if you mess up your fstab at that crunch-point then when you reboot your system will lose track of your /home files! Setting up fstab first and using "sudo mount -a" to mount the partitions checks the modified fstab works at a safe point in the process.
 * Use rsync to move the files
  * [[rsync | Rsync]] is designed for backups of /home, and much more. It is able to maintain other characteristics of the file, like permissions, ownership, and timestamps. There was a lot of debate about which command to copy your files to the new /home partition which stemmed from a time when cp was not able to do it properly (apparently it skipped files?). Cp was not designed to be a powerful backup tool, rsync was. The fix, then, was to use a combination of find and cpio ([[http://www.us.debian.org/doc/manuals/reference/ch-tips.en.html#s-archiving|See section 8.3.5]]).

= The Guide =
Setting up /home on a separate partition is beneficial because your settings, files, and desktop will be maintained if you upgrade, (re)install Ubuntu or another distro. This works because /home has a sub-folder for each user's settings and files which contain all the data & settings of that user. Also, fresh installs for linux typically like to wipe whatever partition they are being installed to so either the data & settings need to be backed-up elsewhere or else avoid the fuss each time by having /home on a different partition.
=== Overview ===

This guide offers detailed instructions for migrating your home directory into its own dedicated partition. Setting up /home on a separate partition is beneficial because your settings, files, and desktop will be maintained if you upgrade, (re)install Ubuntu or another distro. This works because /home has a subdirectory for each user's settings and files which contain all the data & settings of that user. Telling Ubuntu to use an existing home partition can be done by selecting "Manual Partitioning" during the installation of Ubuntu and specifying that you want your home partitions mount point to be /home, '''ensure you mark your /home partition not be formatted in the process'''. You should also make sure the usernames you enter for accounts during installation match usernames that existed in a previous installation.

This guide will follow these 8 basic steps:

 1. Set-up your new partition
 2. Find the uuid (=address) of the new partition
 3. Backup and edit your fstab to mount the new partition as /media/home (just for the time being) and reboot.
 4. Use rsync to migrate all data from /home into /media/home.
 5. Move /home to /old_home to avoid confusion later!
 6. Edit fstab again so the new partition mounts as /home instead of as /media/home
 7. Reboot or remount all. Check system seems to be working well
 8. Delete the /old_home after a while

The guide is written in such a way so that at any point in time if there is a system failure, power outage or random restart that it will not have a negative impact on the system and ''SHOULD'' safeguard against the possibility of the user accidentally deleting their home directory in the process.

= Creating a new partition =
Setting up /home on a separate partition is beneficial because your settings, files, and desktop will be maintained if you upgrade, (re)install Ubuntu or another distro. This works because /home has a subdirectory for each user's settings and files which contain all the data & settings of that user. Also, fresh installs for linux typically like to wipe whatever partition they are being installed to so either the data & settings need to be backed-up elsewhere or else avoid the fuss each time by having /home on a different partition.
Line 16: Line 24:
This is beyond the scope of this page. [[HowtoPartition|Try here if you need help]]. Memorize or write down the location of the partition, something like /sda3. Its been suggested to use either ext2, ext3 or ext4 rather than vfat or ntfs. Using vfat is not supported and may fail, since vfat does not support permissions. Ntfs has an advantage of being journalised, like ext3 & ext4 but it can also be read easily by Windows. It is a lot less stable than ext3 or ext4 so it is really a question of stability compared with easy access for Windows. If you don't dual-boot with Windows or have plenty of hard-drive space then staying with very much safer ext3 or 4 is the obvious choice. This is beyond the scope of this page. [[HowtoPartition|Try here if you need help]]. Memorize or write down the location of the partition, something like /sda3. When you do create a new partition it is highly suggested that you create an ext3 or ext4 partition to house your new home directory.
Line 20: Line 28:
The uuid reference to all partitions get to a [[https://help.ubuntu.com/community/UsingTheTerminal#Starting%20a%20Terminal | command-line]] to try this The uuid (Universally Unique Identifier) reference for all partitions can be found by opening a [[https://help.ubuntu.com/community/UsingTheTerminal#Starting%20a%20Terminal | command-line]] to type the following:
Line 24: Line 32:
Alternatively, for some older releases of Ubuntu the "blkid" command might not work so this could be used instead
===
Alternative Method ===
F
or some older releases of Ubuntu the "blkid" command might not work so this could be used instead
Line 32: Line 42:
Now you just need to note down (copy&paste into a text-file) the uuid of the partition that you have set-up ready to be the new /home partition. Now you just need to take note (copy&paste into a text-file) the uuid of the partition that you have set-up ready to be the new /home partition.
Line 36: Line 46:
These commands should;
 1. Creates a backup of fstab. The "$(date +%Y-%m-%d)" at the end automatically (and quite cleverly) gives the file today's date backwards.
 2. Compares fstab with the new back-up just to be certain it's good.
 3. Opens the standard text-editor (gedit) in Ubuntu to edit fstab. Note that Kubuntu uses "kate" & Xubuntu uses "mousepad" instead but you could use any text-editor you have installed if you prefer.
Your fstab is a file used to tell Ubuntu what partitions to mount at boot. The following commands will duplicate your current fstab, append the year-month-day to the end of the file name, compare the two files and open the original for editing.

1. Duplicate your fstab file:
Line 42: Line 51:
}}}

2. Compare the two files to confirm the backup matches the original:
{{{
Line 43: Line 56:
gksu gedit /etc/fstab
}}}
and add these lines into it
{{{
}}}

3. Open the ori
ginal fstab in a text editor:
{{{
sudo gedit /etc/fstab 
}}} and add these lines into it {{{
Line 48: Line 63:
UUID=???????? /media/home ext3 nodev,nosuid 0 2
}}}
Replace???????? with the UUID number of the intended /home partition.

The Fstab location will be a temporary one - we are to mount the temporary location and copy the existing home to it. (/media/home is assumed in the rest of the guide). If /media/home does not exist yet, then you should create it with the command:
UUID=???????? /media/home ext3 defaults 0 2
}}} and replace the "????????" with the UUID number of the intended /home partition.

NOTE: In the above example, the specified partition in the new text is an ext3, but if yours is an ext4 partition, you should change the part above that says "ext3" to say "ext4", in addition to replacing the ???'s with the correct UUID. Also note that if you are using Kubuntu, Xubuntu or Lubuntu you may need to replace "gedit" with "kate", "mousepad" or "leafpad", respectively. They are text editors included with those distributions.


4. Save and Close the fstab file, then type the following command:
Line 57: Line 74:
Now, mount the partition with: This command will create a new directory, later used for temporarily mounting the new partition. At the end of the procedure this directory can be removed.

Now you can restart your machine or instead of rebooting you might prefer to just re-load the updated fstab
Line 62: Line 81:
Either should have mounted the new partition as /media/home. We will edit the fstab again later so this arrangement of the partition is only temporary.
Line 64: Line 85:
{{{
sudo rsync -axS --exclude='/*/.gvfs' /home/. /media/home/.
}}}
The --exclude='/*/.gvfs' prevents rsync from complaining about not being able to copy .gvfs, but I believe it optional. Even if rsync complains, it will copy everything else anyway. ([[http://ubuntuforums.org/showthread.php?t=791693|See here for discussion on this]])

Next we will copy all files, directories and sub-directories from your current /home directory into the new partition.
If you do not have an encrypted home file system, just do the following:

{{{
sudo rsync -aXS --exclude='/*/.gvfs' /home/. /media/home/.
}}}
I prefer adding the "--progress" tag just before the "--excludes" one - otherwise there is no indication of anything happening. The "--progress" tag reports on each file individually so you see tons of unfamiliar stuff scrolling by very fast. Rsync can be interrupted as many times as you like and it checks to see how much has already been done when you start it up again. So, this copying stage can be broken down into many sessions. After it has completed once it's wise to run it a couple more times just to make sure it includes everything you may have added since first starting the first copying/syncing session - even if you've done the whole thing all in just one session.

The --exclude='/*/.gvfs' prevents rsync from complaining about not being able to copy .gvfs, but I believe it is optional. Even if rsync complains, it will copy everything else anyway. ([[http://ubuntuforums.org/showthread.php?t=791693|See here for discussion on this]])

=== Encrypted file systems ===
If you have an encrypted home file system, then the above will just leave you with an unencrypted copy of your files,
which is probably not what you want. You could re-encrypt them after copying, or copy them in their encrypted form.
Here is one way to do that.

First, you'll need to shut down, and reboot from a LiveCD or USB stick. Then you'll need to mount your root partition
and new home partition. (You can do this by selecting those devices in the file viewer). They will be mounted under
/media/ubuntu - so for example, if you named your root partition linux-root, then your old home directory will be found at
/media/ubuntu/linux-root/home. And if you named your new home partition linux-home, then this will be found at
/media/ubuntu/linux-home. So, now you can copy your encrypted home files
(here assuming your partitions are named linux-root and linux-home):

{{{
sudo rsync -aXS /media/ubuntu/linux-root/home/. /media/ubuntu/linux-home/.
}}}
There is no point trying to exclude any files with specific names, because the names of the files are encrypted too!

Leave your machine running from the LiveCD or USB for the moment.
Line 70: Line 117:
The data in /home now exists in 2 places but we need to check that it looks about right. If it looks approximately the same then it is probably completely right.


== A Sneaky Safety Manoeuvre ==
So you now have 2 copies of your /home folder. The new one on the new partition and the old one still in the same partition it was always in. The problem is how to delete the right one!! Since we are still using the old /home 'just' rename your current /home folder:
You should now have two duplicate copies of all the data within your home directory; the original being located in /home and the new duplicate located in /media/home. You should confirm all files and directories copied over successfully. One way to do this (for an unencrypted file system) is by using the diff command:

{{{
sudo diff -r /home /media/home -x ".gvfs/*"
}}}

If you are doing this from a LiveCd or to an existing partition that already has stuff on it you may find differences but hopefully it should be obvious which diffs you can ignore.

You can also expect to see some errors about files not found. These are due to symbolic links that point to places that don't presently exist (but will do after you have rebooted). You can ignore these - but check out anything else.

=== Encrypted file systems ===
If you have an encrypted file system, the command will look more like this.
{{{
sudo diff -r /media/ubuntu/linux-root/home /media/ubuntu/linux-home
}}}

Now you can shut-down, remove the LiveCD / USB stick, and reboot as normal.


== Moving /home into /old_home ==

Backing up your old home, just in case things have not gone completely smoothly, is best done right now. Here is how:

As long as you have not rebooted yet, you will still see 2 copies of your /home directory; the new one on the new partition (currently mounted as /media/home) and the old one still in the same partition it was always in (currently mounted as /home). We need to move the contents of the old home directory out of the way and create an empty "place-holder" directory to act as a "mount point" for our new partition.

Type the following string of commands in to do all this at once:
{{{
cd / && sudo mv /home /old_home && sudo mkdir /home
}}}

By default, when you open a terminal window it places you within your home directory. Typing cd / takes us to the root directory and out of home so we can then use the sudo mv command to essentially rename /home into /old_home, and finally create a new, empty /home placeholder.


== Preparing fstab for the switch ==
We now need to modify the fstab again to point to the new partition and mount it as /home. So again on a command-line
{{{
sudo gedit /etc/fstab
}}}
and now edit the lines you added earlier, changing the "/media/home" part to simply say "/home" so that it looks like this:
{{{
# (identifier) (location, eg sda5) (format, eg ext3 or ext4) (some settings)
UUID=???????? /home ext3 defaults 0 2
}}}

Then, press Save, close the file but don't reboot just yet.


== Reboot or Remount all ==
With;
 1. your fstab now edited to mount your new partition to our /home place-holder and
 2. the original /home now called /old_home,

it should be a good time to reboot your computer to check the whole thing really did work. Your new partition should mount as /home and everything should look exactly the same as it did before you started.

Btw, geeks might prefer to avoid rebooting by just re-loading the updated fstab
{{{
sudo mount -a
}}}
There is no need to reboot - unless you have an encrypted file system.


=== Troubleshooting ===
If you receive an error like 'The volume may already be mounted', use the following command to unmount the drive first before re-doing the last step again. (note the "n" should be missing from the command, making it "umount")
{{{
sudo umount /media/home/
}}}

Then try mounting again
{{{
sudo mount -a
}}}

== Deleting the old Home ==

You can delete your old home directory with:
Line 77: Line 194:
sudo mv /home /old_home
}}}
Now re-create a new blank /home directory just in case it has all gone wrong!
{{{
cd /
sudo mkdir -p /home/user
}}}
Now if things have gone wrong then the system will boot into what appears to be a fresh install with all your data missing. In fact, of course, there are still 2 copies of it all :)

== Make the Switch ==
We now need to modify the fstab again to point to the new /home and mount it properly. So again on a command-line
{{{
gksu gedit /etc/fstab
}}}
and now edit the lines you added earlier to delete the /media part. This should make /media/home into /home as follows
{{{
# (identifier) (location, eg sda5) (format, eg ext3 or ext4) (some settings)
UUID=???????? /home ext3 nodev,nosuid 0 2
}}}

and finally, remount the partition with:
{{{
sudo mount -a
}}}

Does it all still look fine? Can you still access your files & folders from the "Places" menu? Reboot to ensure that currently running programs use the /home that is on the new partition.

== After a reboot ==
If everything is working, great, you can delete /old_home. If things aren't working, you can undo by moving your /old_home back to /home, and delete the line you added to fstab.

sudo rm -r /old_home
}}}
Be careful with the above command as mistyping it could result in the deletion of other files and directories.
Line 114: Line 203:

== Different filesystems on the same disk ==

If you're moving from Windows and your new home partition is going to be an old ntfs partition (the D: disk) while you convert the C: disk to a journaling partition where you install Linux, this won't work, there will be a huge load on the processor. You should convert the two partitions to ext3 or ext4 or keep both partitions as ntfs (I haven't checked this last option). But working with two different filesystems on the same drive simultaneously doesn't seem to be a good option.

Overview

This guide offers detailed instructions for migrating your home directory into its own dedicated partition. Setting up /home on a separate partition is beneficial because your settings, files, and desktop will be maintained if you upgrade, (re)install Ubuntu or another distro. This works because /home has a subdirectory for each user's settings and files which contain all the data & settings of that user. Telling Ubuntu to use an existing home partition can be done by selecting "Manual Partitioning" during the installation of Ubuntu and specifying that you want your home partitions mount point to be /home, ensure you mark your /home partition not be formatted in the process. You should also make sure the usernames you enter for accounts during installation match usernames that existed in a previous installation.

This guide will follow these 8 basic steps:

  1. Set-up your new partition
  2. Find the uuid (=address) of the new partition
  3. Backup and edit your fstab to mount the new partition as /media/home (just for the time being) and reboot.
  4. Use rsync to migrate all data from /home into /media/home.
  5. Move /home to /old_home to avoid confusion later!
  6. Edit fstab again so the new partition mounts as /home instead of as /media/home
  7. Reboot or remount all. Check system seems to be working well
  8. Delete the /old_home after a while

The guide is written in such a way so that at any point in time if there is a system failure, power outage or random restart that it will not have a negative impact on the system and SHOULD safeguard against the possibility of the user accidentally deleting their home directory in the process.

Creating a new partition

Setting up /home on a separate partition is beneficial because your settings, files, and desktop will be maintained if you upgrade, (re)install Ubuntu or another distro. This works because /home has a subdirectory for each user's settings and files which contain all the data & settings of that user. Also, fresh installs for linux typically like to wipe whatever partition they are being installed to so either the data & settings need to be backed-up elsewhere or else avoid the fuss each time by having /home on a different partition.

Setup Partitions

This is beyond the scope of this page. Try here if you need help. Memorize or write down the location of the partition, something like /sda3. When you do create a new partition it is highly suggested that you create an ext3 or ext4 partition to house your new home directory.

Find the uuid of the Partition

The uuid (Universally Unique Identifier) reference for all partitions can be found by opening a command-line to type the following:

sudo blkid

Alternative Method

For some older releases of Ubuntu the "blkid" command might not work so this could be used instead

sudo vol_id -u <partition>

for example

sudo vol_id -u /dev/sda3

Now you just need to take note (copy&paste into a text-file) the uuid of the partition that you have set-up ready to be the new /home partition.

Setup Fstab

Your fstab is a file used to tell Ubuntu what partitions to mount at boot. The following commands will duplicate your current fstab, append the year-month-day to the end of the file name, compare the two files and open the original for editing.

1. Duplicate your fstab file:

sudo cp /etc/fstab /etc/fstab.$(date +%Y-%m-%d)

2. Compare the two files to confirm the backup matches the original:

cmp /etc/fstab /etc/fstab.$(date +%Y-%m-%d)

3. Open the original fstab in a text editor:

sudo gedit /etc/fstab 

and add these lines into it

# (identifier)  (location, eg sda5)   (format, eg ext3 or ext4)      (some settings) 
UUID=????????   /media/home    ext3          defaults       0       2 

and replace the "????????" with the UUID number of the intended /home partition.

NOTE: In the above example, the specified partition in the new text is an ext3, but if yours is an ext4 partition, you should change the part above that says "ext3" to say "ext4", in addition to replacing the ???'s with the correct UUID. Also note that if you are using Kubuntu, Xubuntu or Lubuntu you may need to replace "gedit" with "kate", "mousepad" or "leafpad", respectively. They are text editors included with those distributions.

4. Save and Close the fstab file, then type the following command:

sudo mkdir /media/home

This command will create a new directory, later used for temporarily mounting the new partition. At the end of the procedure this directory can be removed.

Now you can restart your machine or instead of rebooting you might prefer to just re-load the updated fstab

sudo mount -a

Either should have mounted the new partition as /media/home. We will edit the fstab again later so this arrangement of the partition is only temporary.

Copy /home to the New Partition

Next we will copy all files, directories and sub-directories from your current /home directory into the new partition. If you do not have an encrypted home file system, just do the following:

sudo rsync -aXS --exclude='/*/.gvfs' /home/. /media/home/.

I prefer adding the "--progress" tag just before the "--excludes" one - otherwise there is no indication of anything happening. The "--progress" tag reports on each file individually so you see tons of unfamiliar stuff scrolling by very fast. Rsync can be interrupted as many times as you like and it checks to see how much has already been done when you start it up again. So, this copying stage can be broken down into many sessions. After it has completed once it's wise to run it a couple more times just to make sure it includes everything you may have added since first starting the first copying/syncing session - even if you've done the whole thing all in just one session.

The --exclude='/*/.gvfs' prevents rsync from complaining about not being able to copy .gvfs, but I believe it is optional. Even if rsync complains, it will copy everything else anyway. (See here for discussion on this)

Encrypted file systems

If you have an encrypted home file system, then the above will just leave you with an unencrypted copy of your files, which is probably not what you want. You could re-encrypt them after copying, or copy them in their encrypted form. Here is one way to do that.

First, you'll need to shut down, and reboot from a LiveCD or USB stick. Then you'll need to mount your root partition and new home partition. (You can do this by selecting those devices in the file viewer). They will be mounted under /media/ubuntu - so for example, if you named your root partition linux-root, then your old home directory will be found at /media/ubuntu/linux-root/home. And if you named your new home partition linux-home, then this will be found at /media/ubuntu/linux-home. So, now you can copy your encrypted home files (here assuming your partitions are named linux-root and linux-home):

sudo rsync -aXS /media/ubuntu/linux-root/home/. /media/ubuntu/linux-home/.

There is no point trying to exclude any files with specific names, because the names of the files are encrypted too!

Leave your machine running from the LiveCD or USB for the moment.

Check Copying Worked

You should now have two duplicate copies of all the data within your home directory; the original being located in /home and the new duplicate located in /media/home. You should confirm all files and directories copied over successfully. One way to do this (for an unencrypted file system) is by using the diff command:

sudo diff -r /home /media/home -x ".gvfs/*"

If you are doing this from a LiveCd or to an existing partition that already has stuff on it you may find differences but hopefully it should be obvious which diffs you can ignore.

You can also expect to see some errors about files not found. These are due to symbolic links that point to places that don't presently exist (but will do after you have rebooted). You can ignore these - but check out anything else.

Encrypted file systems

If you have an encrypted file system, the command will look more like this.

sudo diff -r /media/ubuntu/linux-root/home /media/ubuntu/linux-home

Now you can shut-down, remove the LiveCD / USB stick, and reboot as normal.

Moving /home into /old_home

Backing up your old home, just in case things have not gone completely smoothly, is best done right now. Here is how:

As long as you have not rebooted yet, you will still see 2 copies of your /home directory; the new one on the new partition (currently mounted as /media/home) and the old one still in the same partition it was always in (currently mounted as /home). We need to move the contents of the old home directory out of the way and create an empty "place-holder" directory to act as a "mount point" for our new partition.

Type the following string of commands in to do all this at once:

cd / && sudo mv /home /old_home && sudo mkdir /home

By default, when you open a terminal window it places you within your home directory. Typing cd / takes us to the root directory and out of home so we can then use the sudo mv command to essentially rename /home into /old_home, and finally create a new, empty /home placeholder.

Preparing fstab for the switch

We now need to modify the fstab again to point to the new partition and mount it as /home. So again on a command-line

sudo gedit /etc/fstab

and now edit the lines you added earlier, changing the "/media/home" part to simply say "/home" so that it looks like this:

# (identifier)  (location, eg sda5)   (format, eg ext3 or ext4)      (some settings) 
UUID=????????   /home    ext3          defaults       0       2

Then, press Save, close the file but don't reboot just yet.

== Reboot or Remount all == With;

  1. your fstab now edited to mount your new partition to our /home place-holder and
  2. the original /home now called /old_home,

it should be a good time to reboot your computer to check the whole thing really did work. Your new partition should mount as /home and everything should look exactly the same as it did before you started.

Btw, geeks might prefer to avoid rebooting by just re-loading the updated fstab

sudo mount -a

There is no need to reboot - unless you have an encrypted file system.

Troubleshooting

If you receive an error like 'The volume may already be mounted', use the following command to unmount the drive first before re-doing the last step again. (note the "n" should be missing from the command, making it "umount")

sudo umount /media/home/

Then try mounting again

sudo mount -a

Deleting the old Home

You can delete your old home directory with:

cd /
sudo rm -r /old_home

Be careful with the above command as mistyping it could result in the deletion of other files and directories.

Technical Notes and Resources

Rsync was chosen over cp and find|cpio because it seemed to maintain permissions.

http://ubuntu.wordpress.com/2006/01/29/move-home-to-its-own-partition/

http://ubuntuforums.org/showthread.php?t=46866

Different filesystems on the same disk

If you're moving from Windows and your new home partition is going to be an old ntfs partition (the D: disk) while you convert the C: disk to a journaling partition where you install Linux, this won't work, there will be a huge load on the processor. You should convert the two partitions to ext3 or ext4 or keep both partitions as ntfs (I haven't checked this last option). But working with two different filesystems on the same drive simultaneously doesn't seem to be a good option.

Partitioning/Home/Moving (last edited 2015-11-13 13:16:06 by 80)