Contents

  1. Persistent live systems
  2. mkusb / dus
  3. The classic dus-persistent method is described here
    1. Method developed from 'grub-n-iso'
      1. Cloning the head end of the drive - using 'grub.img'
    2. Using data from the source iso file
    3. Compressed image file with a persistent live system
      1. Download links
    4. Small 9w systems with guidus alias mkusb-dus and gparted installed
    5. Xubuntu Core 20.04 LTS with mkusb 12.4.5 (mkusb-dus and mkusb-plug)
    6. mkusb 12.5.0 makes a nice grub menu
    7. Link to mkusb page about Knoppix
    8. Partitions
    9. Comparison with 'live only' systems
    10. Advantages
    11. Disadvantages
  4. Alternatives
  5. 'Download and install automatically' can break persistent live media
    1. Discussion
    2. Select 'Display immediately', if your flavour and version is affected
    3. Solution for iso file versions that are affected
      1. Change the default settings by mkusb
      2. Change the default settings in the Ubuntu flavours
  6. Backup and restore of persistent overlay data
    1. Update and upgrade
      1. Common experience
      2. Test with update and full-upgrade
      3. home directory survives
    2. Backup & restore of the whole casper-rw partition
      1. Backup example
      2. Restore example
      3. Automatic tarball names
      4. Copy the tarball to another drive
      5. Select another directory/file
      6. Screenshots
    3. Backup & restore of the home directory in a casper-rw partition
      1. Upgrade (or repair) a persistent live system like this
      2. Tests
      3. Notes about backup & restore of the home directory
      4. Screenshots
  7. Installing - using the Install icon on the desktop
    1. The installer Ubiquity in most Ubuntu flavours
      1. Boot persistent live to RAM
    2. The installer Calamares in Lubuntu 20.04

Return to the main mkusb page


Persistent live systems

Most of the time it is enough with a live only USB pendrive and only a waste of effort to create a persistent live system. The cloning method of mkusb is very reliable and works with most linux distros. It needs hybrid iso files. But sometimes it is better to have a persistent live system, and beginning with version 10 mkusb can create such systems.

mkusb / dus

mkusb version 10-11, old:

../pictures/persist-01-persistence-selected-small.png

mkusb version 12, dus:

../pictures/persist-01-select-persistence-1202-small.png

Select dus-iso2usb or dus-persistent:

select_method-tool_persist_lubuntux.png


The classic dus-persistent method is described here

... and dus-iso2usb is described at the following link: ../dus-iso2usb.


../pictures/dus-24-persistent-live-system_quick-start-manual.png

Method developed from 'grub-n-iso'

This persistent live system is developed from booting via grub and the ISO file 'grub-n-iso' by Andre Rodovalho, described in

One pendrive for all PC (Intel/AMD) computers - single-boot dual-boot multi-boot and

How to Create a EFI/UEFI GRUB2 Multiboot USB drive to boot ISO images

The same pendrive can boot in UEFI mode and BIOS mode with a 32-bit operating system. Today 64-bit operating systems can boot almost all except the oldest computers. But this persistent live system cannot boot with secure boot. Data from usb-pack_efi.tar.gz are used to boot.

Cloning the head end of the drive - using 'grub.img'

The program package grub-pc cannot be used in installed systems running in UEFI mode. This means that the BIOS bootloader cannot be installed with the grub-install program. Instead, a compressed image file was made from a system created where is was possible, grub.img.xz. This image contains the first mibibyte of the drive plus the bios_grub and the usbboot partitions. The usbboot partition is also an efi partition for booting in UEFI mode.

This feature is introduced in mkusb version 12.2.7 and some details are described at ../12#Improvements_and_new_features

Using data from the source iso file

The grub boot system can be copied from an amd64 iso file. It can be modified slightly (to make a menuentry for persistence and to point to another partition). This works with the amd64 (64-bit) systems based on Ubuntu also with secure boot.

Persistent live systems made with this method boot in UEFI mode and can be made to boot in BIOS mode, if the program grub-install for BIOS is available. And it is, except in installed systems running in UEFI mode, where the package grub-pc cannot be installed without conflict with existing EFI grub packages. So this method provides a tool the make persistent live systems in UEFI mode and for UEFI mode (for amd64 systems).

In a live or persistent live system it is 'always' possible to install grub-pc, so it is possible to create systems that boot also in BIOS mode from a persistent live system running in UEFI mode.

Compressed image file with a persistent live system

Use a cloning tool, for example Disks or mkusb in linux, or 7-zip and Win32 Disk Imager in Windows to expand and {copy/clone/flash/burn/restore/install} this kind of files into a USB pendrive, a flash memory card or a hard disk drive or SSD with at least 8 GB. You can expand one or more partitions in order to use the whole drive, if it is bigger than 8 GB. This is described in the following link: growit.pdf; alternate link: ../../Installation/UEFI-and-BIOS/GrowIt.pdf

Short explanation: The original reason to create such images with mkusb was that, if the computer runs an installed Ubuntu system in UEFI mode, it will not be possible to install and use the package 'grub-pc' to make the target persistent live drive bootable in BIOS mode (because there is a conflict with 'grub-efi'). But in a persistent live Ubuntu, both 'grub-pc' and 'grub-efi' can be installed alongside each other. So it is possible to create systems that boot also in BIOS mode from a persistent live system running in UEFI mode. The current version of mkusb can work around that problem by cloning a small image of the head end of a drive, that boot in UEFI and BIOS mode.

But there can still be a reason to use these compressed images: They can be used also from Windows and be installed into a USB pendrive (by expansion and cloning).

There is a question "Why are img files provided for mkusb persistent but only iso file are allowed?" and an answer at AskUbuntu: with a more detailed explanation.

Download one of the following compressed image files from one of these links,

drive.google.com/open?id=0BzX-18u3W1sQUXI2YV95dHJxVlE

phillw.net/isos/linux-tools/uefi-n-bios

If one link does not work, please try the other one.

Small 9w systems with guidus alias mkusb-dus and gparted installed

If the explicit link (to http://phillw.net/isos/linux-tools/9w) fails, please try drive.google.com/open?id=0BzX-18u3W1sQUXI2YV95dHJxVlE

There are small files, only 230 MiB, as iso files, compressed image files and self-extracting exe files, that can provide systems with mkusb version 12 alias dus, guidus (June 2 2017). These systems can be cloned from their files to USB pendrives or memory cards in Windows as well as linux. These are 32-bit systems, that normally only boot in BIOS mode. One 9w image can boot in UEFI mode (but not with secure boot). It is a persistent live system using usb-pack-efi to allow booting also in UEFI mode.

See the following links

9W/Small 9w iso file with guidus and gparted installed

http://phillw.net/isos/linux-tools/9w/

Win32DiskImager/compressed-image_2_USB-or-SD

9W-DUS_i686.exe

These 9w systems can be used to create live-only as well as persistent live drives with Ubuntu and Debian Jessie.

../pictures/niow-12_9w-dus12.1.9-screenshot_800x600.png

../pictures/niow-13_9w-dus12.1.9-screenshot_work-done.png

Xubuntu Core 20.04 LTS with mkusb 12.4.5 (mkusb-dus and mkusb-plug)

New Xubuntu Core and new mkusb

xubuntu-core_w_mkusb.png

mkusb 12.5.0 makes a nice grub menu

In mkusb 12.5.0 dus-persistent makes nice grub menus. The text size is controlled and there is a wallpaper. The menu looks the same in BIOS mode and UEFI mode. In high resolution screens the menu text was too small in UEFI mode, and it was difficult to read. Please notice that this feature works for Ubuntu and the Ubuntu family flavours with the default settings, but it does not work with usb-pack-efi. See this screenshot.

35-mkusb-dus-persistent-with-wallpaper1.png

The following link describes how to make a Knoppix system with mkusb, ../install-to-debian#Knoppix

Partitions

When you select persistent mkusb creates a partition for persistence in the target drive and the target system can use it automatically. It creates totally five partitions automatically for you.

In a separate menu window you can choose between a GUID partition table, GPT, and an old style MSDOS partition table. Normally GPT is recommended (and it works with huge drives (greater than 2 TB), but many HP computers need an MSDOS partition table to boot directly from USB.

The first partition, labeled 'usbdata', is actually located at the end of the drive, but it has number one because several Windows versions will only read the first partition (of the partition table) on USB drives. This partition has an NTFS file system, and can be used both when connected to a computer running linux and Windows, so it is suitable for transfer of files as well as a common storage.

The second partition is used to store program code for the grub BIOS bootloader if a GUID partition table, GPT, and as an extended partition, a container for logical partitions if an MSDOS partition table.

The persistent live drive boots from the third partition with a FAT32 alias vfat file system.

The operating system is cloned from the iso file to the fourth partition.

The fifth partition with an ext4 file system stores the overlay data for persistence.

../pictures/persist-05-mkusb-console-only-partitions-3.png

Comparison with 'live only' systems

Otherwise, when you select live only mkusb clones the iso file and there will be no persistence (except with some openSUSE iso files, that make persistent live systems as cloned). This kind of cloned system with an Ubuntu 64-bit operating system can boot also in secure mode.

Advantages

Disadvantages

Alternatives

Unetbootin and the 'Ubuntu Startup Disk Creator' alias usb-creator 0.2.x can also be used, when you want persistence in the target drive. They create and use a casper-rw file with the maximum size 4 GB (due to the file system FAT32). (But usb-creator version 0.3.2 in Xenial Xerus is cloning like mkusb makes live-only drives.)

Persistence works with an ext2, ext3 or ext4 partition with the label casper-rw, and that partition can be 'anywhere', in another pendrive or in an internal drive (which makes it faster). Persistence with an ext partition 'anywhere' works when booting from a DVD as well as from a cloned iso file (from a pendrive made by mkusb). Just add the boot option persistent when you want persistence. Then the live system will search for a casper-rw file or casper-rw partition.

Debian Jessie uses the partition label or file name persistence and ToriOS uses live-rw (instead of casper-rw, and this is managed by mkusb).

You can also create a home-rw partition to store the content of your home directory and it will survive even if the system modifications are damaged in the casper-rw partition.

'Download and install automatically' can break persistent live media

Look at the persistent live Ubuntu 16.04 LTS system - the 'Software & Updates' screen / 'Update' - and notice, that 'Automatic updates' is set to Download and install automatically alias 'unattended upgrades'. This can create severe problems in a persistent live system.

After leaving the persistent live Ubuntu 16.04 LTS system running overnight, it had performed an automatic upgrade:

df revealed that the content in casper-rw had increased to 1.6 GiB (in a fresh system with an almost empty casper-rw partition).

Discussion

If not enough space (and inodes) in the casper-rw partition or file, the upgrade operation will be incomplete and fail. There might be big problems even if there is space left in casper-rw. Finally, it is wasting the available space, if you have a small casper-rw partition or casper-rw file.

This is caused by a change of the default action, when there are security updates. A survey indicates that this default setting is different between the flavours and versions of Ubuntu. Lubuntu keeps the setting 'Display immediately', while the other tested flavours change it from 14.04 LTS to 16.04 LTS.

Select 'Display immediately', if your flavour and version is affected

Until this is resolved, it is a good idea to select 'Display immediately' via the graphical interface, but above all, to take regular backups, when you use a persistent live system. If you want more details, see the following link with screenshots from different Ubuntu flavours and versions:

Re: Unattended-upgrades can break persistent live media

Solution for iso file versions that are affected

Change the default settings by mkusb

A method to change from 'Download and install automatically' to 'Display immediately' was tested in mkusb version 11.0.2, and is available in the current versions of mkusb. See ../v11.

Change the default settings in the Ubuntu flavours

The bug report Bug #1619188: Unattended upgrades can break persistent live media was created at Launchpad in order to change the default settings for security updates for persistent live and live-only systems. The current setting 'Download and install automatically' is good for installed systems. The bug affects Ubuntu 16.04 and 16.04.1. It is squashed in Ubuntu 16.04.2 LTS.

Backup and restore of persistent overlay data

The partition that stores persistent overlay data is easily damaged, particularly if you remove the pendrive before it is unmounted.

Update and upgrade

It is a good idea to use persistence to add functionality by installing packages and tweak the system to what you like.

Common experience

But it is bad idea to update/upgrade a persistent live system continuously. Then you will soon get a borked system according to the experience of several people.

Test with update and full-upgrade

I have tested (and am still testing) a persistent live Lubuntu 16.04.1 LTS system in a 60 GB SSD. Compared to a typical USB pendrive the SSD is fast, big and reliable. It was possible to update and upgrade it to be up to date in June 2018, which is almost 2 years after the iso file was released, and the system works after several reboots and installation of new program packages.

sudo apt update
sudo apt full-upgrade

home directory survives

If/when the persistence no longer works you may need to start all over again by removing what has been stored. Often you can keep the [overlay copy of the] home directory (and delete all other subdirectories of your overlay system in the casper-rw partition). Delete while running it live-only! This will save many settings, tweaks and personal files, but you must reinstall the program packages, that have been added.

Backup & restore of the whole casper-rw partition

Therefore it is important to have a convenient way to backup and restore the persistent data. This was introduced in mkusb 10.3.4 with shell-scripts to make it easy to backup and restore the overlayed persistent data of the persistent live system created by mkusb. These shell-scripts are written to

These scripts work in linux operating systems based on Ubuntu. In some versions, they are not executable, but can be run as parameter to bash. Boot into the live-only mode and run the scripts according to the following examples. (The scripts will complain if you try to use them in persistent mode.)

Backup example

cd /media/xubuntu/usbdata
ls -l
bash backup

Restore example

cd /media/xubuntu/usbdata
ls -l
bash restore

Automatic tarball names

The name of the tarball file (containing the backup) is updated automatically to create versions. Remove old versions manually!

1-casper-rw.tar.gz
2-casper-rw.tar.gz
3-casper-rw.tar.gz
...

Copy the tarball to another drive

Finally, to be sure the backup survives, you should copy the tarball to another drive, that you keep somewhere else, so that both backup copies are not lost. If you use the pendrive regularly, you should make backup copies of the casper-rw partition regularly too, maybe once a week or twice a month.

Select another directory/file

It is also possible to select another directory where to write/read the tarball. In that case, you add a parameter to the command line, for example

bash backup /media/xubuntu/backup  # target directory in another drive, a 'backup' drive

bash restore /media/xubuntu/backup/1-casper-rw.tar.gz  # source tarball in another drive, 'backup'

The commands under the hood of backup and restore are described in this post in the Ubuntu Forums.

Screenshots

See the whole screenshots at /LXLE#Backup_and_Restore

../pictures/persist-26-LXLE_16_04_02_32_persistent_live_backup-and-restore_after-backup_truncated.png

../pictures/persist-26-LXLE_16_04_02_32_persistent_live_backup-and-restore_after-restore_truncated.png

Backup & restore of the home directory in a casper-rw partition

New in version mkusb 12.3.2

The whole casper-rw partition can be restored to a persistent live drive made from the same iso file. But if you want to upgrade the persistent live system from a new daily iso file of an LTS system or to another version, it will not work. The overlay is not compatible with the system from from iso file.

However, if you backup the home directory in a casper-rw partition. you can restore it to a persistent live system from a new daily iso file of an LTS system or to another version (newer or older). So you can upgrade (and downgrade) a persistent live system including the data files, settings and tweaks in your home directory. But you must reinstall programs that you installed manually, and re-do global tweaks, if any.

This was introduced in mkusb 12.3.2 with shell-scripts to make it easy to backup and restore the overlayed persistent data of the persistent live system created by mkusb. These shell-scripts are merged into mkusb-dus. There is a detailed description at the following link to the Ubuntu Forums Tutorial "One pendrive for all PC (Intel/AMD) computers - single-boot dual-boot multi-boot"

Post #149 - #156: Backup and restore the /home directory in casper-rw partitions of mkusb persistent drives

There are two modes, to

The purpose of this tool is to make it easy to keep the persistent live drive up to date. It is usually possible to keep the home directory including personal files and the settings (usually in hidden configuration files in the user's home directory).

Upgrade (or repair) a persistent live system like this

Tests

This method works with standard Ubuntu desktop and the Ubuntu community flavours. It is tested with

Notes about backup & restore of the home directory

Screenshots

Select backup or restore persistent live home from the starter menu "Install / Backup / Restore / Wipe"

29-install-backup-restore-wipe_Backup-persistent-live-home.png

Select persistent live drive, if more than one is connected

08-mkusb-backup-n-restore-home.png

Link to more screenshots ../pictures#Backup-n-restore-home

Installing - using the Install icon on the desktop

Normally you would create a live-only USB drive, if you 'only' want to install your linux system, but the Install icon on the desktop is available also in a persistent live system.

The installer Ubiquity in most Ubuntu flavours

The installer icon starts the ubiquity installer, and there can be problems. The installer might be confused by the partition structure of the persistent live system created by mkusb.

Boot persistent live to RAM

This method works with standard Ubuntu, most Ubuntu flavours and older versions of persistent live Lubuntu:

- Boot a persistent live system with to RAM

- If you want to install with encrypted home, you should swapoff everything (including zRAM) before starting the installer

sudo swapoff -a

- Start the installer

- Let the installer unmount all partitions that it wants to unmount (including on the boot drive; this is possible when using to RAM)

The installer Calamares in Lubuntu 20.04

The installer icon starts the installer Calamares, and there can be problems (different from those of Ubiquity).

- Calamares works from live (live-only) as well as from persistent live systems made by mkusb.

- Calameres does not work from live (live-only) systems booted to RAM using the boot option 'toram'. See this bug report.


Return to the main mkusb page

mkusb/persistent (last edited 2022-09-01 13:31:22 by nio-wiklund)