Xen is a type 1, bare-metal virtual machine monitor (or hypervisor), which provides the ability to run one or more operating system instances on the same physical machine. Xen, like other types of virtualization, is useful for many use cases such as server consolidation and isolation of production and development environments (e.g. corporate and personal environments on the same system).
As of Ubuntu 11.10 (Oneiric), the default kernel included in Ubuntu can be used directly with the Xen hypervisor as the management (or control) domain (Dom0 or Domain0 in Xen terminology).
The rest of this guide gives a basic overview of how to set up a basic Xen system and create simple guests. Our example uses LVM for virtual disks and network bridging for virtual network cards. It also assumes Xen 4.1 (the version available in 12.04) and the xend toolstack. It assumes a familiarity with general virtualization issues, as well as with the specific Xen terminology. Please see the Xen wiki for more information.
During installation of Ubuntu
During the install of Ubuntu for the Partitioning method choose "Guided - use the entire disk and setup LVM". Then, when prompted to enter "Amount of volume group to use for guided partitioning:" Enter a value just large enough for the Xen Dom0 system, leaving the rest for virtual disks. Enter a value smaller than the size of your installation drive. For example 10 GB or even 5 GB should be large enough for a minimal Xen Dom0 system. Entering a percentage of maximum size (e.g. 25%) is also a reasonable choice.
Install a 64-bit hypervisor. (A 64-bit hypervisor works with a 32-bit dom0 kernel, but allows you to run 64-bit guests as well.)
sudo apt-get install xen-hypervisor-amd64
Modify GRUB to default to booting Xen ("Xen 4.1-amd64" should be replaced with the appropriate name, in 12.10 the line is "Ubuntu GNU/Linux, with Xen hypervisor". The current string can be obtained by looking for one of the menuentry lines in /boot/grub/grub.cfg. In theory the first element created by the 20_linux_xen script):
sudo sed -i 's/GRUB_DEFAULT=.*\+/GRUB_DEFAULT="Xen 4.1-amd64"/' /etc/default/grub sudo update-grub
Set the default toolstack to xm (aka xend):
sudo sed -i 's/TOOLSTACK=.*\+/TOOLSTACK="xm"/' /etc/default/xen
And then verify that the installation has succeeded:
sudo xm list Name ID Mem VCPUs State Time(s) Domain-0 0 945 1 r----- 11.3
This section describes how to set up linux bridging in Xen. It assumes eth0 is both your primary interface to dom0 and the interface you want your VMs to use. It also assumes you're using DHCP.
sudo apt-get install bridge-utils
Note if you are working with a desktop install, disable Network Manager.
sudo update-rc.d network-manager disable sudo /etc/init.d/network-manager stop
Edit /etc/network/interfaces, and make it look like this:
auto lo iface lo inet loopback auto xenbr0 iface xenbr0 inet dhcp bridge_ports eth0 auto eth0 iface eth0 inet manual
Restart networking to enable xenbr0 bridge:
sudo /etc/init.d/networking restart
The brctl command is useful for providing addition bridge information. See: man brctl
There are many options for installing guest images:
xen-tools: A set of scripts for creating various PV guests
virt-manager.org: A management system using libvirt
- Converting an existing installation
Downloading pre-build guest images (e.g. http://wiki.xen.org/wiki/Guest_VM_Images)
Or you can manually create one, as described below.
Manually creating a PV Guest VM
In this section we will focus on Paravirtualized (or PV) guests. PV guests are guests that are made Xen-aware and therefore can be optimized for Xen.
As a simple example we'll create a PV guest in LVM logical volume (LV) by doing a network installation of Ubuntu (other distros such as Debian, Fedora, and CentOS can be installed in a similar way).
choose your volume group (VG)
sudo lvcreate -L 4G -n ubuntu /dev/<VG>
get netboot images
choose an archive mirror https://launchpad.net/ubuntu/+archivemirrors
sudo mkdir -p /var/lib/xen/images/ubuntu-netboot cd /var/lib/xen/images/ubuntu-netboot sudo wget <mirror>/ubuntu/dists/precise/main/installer-amd64/current/images/netboot/xen/initrd.gz wget <mirror>/ubuntu/dists/precise/main/installer-amd64/current/images/netboot/xen/vmlinuz
With a specific mirror chosen:
sudo mkdir -p /var/lib/xen/images/ubuntu-netboot cd /var/lib/xen/images/ubuntu-netboot sudo wget http://mirror.anl.gov/pub/ubuntu/dists/precise/main/installer-amd64/current/images/netboot/xen/initrd.gz sudo wget http://mirror.anl.gov/pub/ubuntu/dists/precise/main/installer-amd64/current/images/netboot/xen/vmlinuz
Set up the initial guest configuration: /etc/xen/ubuntu.cfg
name = "ubuntu" memory = 256 disk = ['phy:/dev/<VG>/ubuntu,xvda,w'] vif = [' '] kernel = "/var/lib/xen/images/ubuntu-netboot/vmlinuz" ramdisk = "/var/lib/xen/images/ubuntu-netboot/initrd.gz" extra = "debian-installer/exit/always_halt=true -- console=hvc0"
Start the VM and connect to console (-c):
sudo xm create /etc/xen/ubuntu.cfg -c
Do the install.
Once installed, we can use pygrub as the bootloader.
sudo ln -s /usr/lib/xen-4.1/bin/pygrub /usr/bin/pygrub
Once the install is done, the VM will shutdown. Next change the guest config, /etc/xen/ubuntu.cfg:
name = "ubuntu" memory = 256 disk = ['phy:/dev/<VG>/ubuntu64,xvda,w'] vif = [' '] bootloader = "pygrub" #kernel = "/var/lib/xen/images/ubuntu-netboot/amd64/vmlinuz" #ramdisk = "/var/lib/xen/images/ubuntu-netboot/amd64/initrd.gz" #extra = "debian-installer/exit/always_halt=true -- console=hvc0"
Start the VM and connect to console (-c):
sudo xm create /etc/xen/ubuntu.cfg -c
Manually installing an HVM Guest VM
Download Install ISO.
choose your VG
Create a LV
sudo lvcreate -L 4G -n ubuntu-hvm /dev/<VG>
Create a guest config file /etc/xen/ubuntu-hvm.cfg
builder = "hvm" name = "ubuntu-hvm" memory = "512" vcpus = 1 vif = [''] disk = ['phy:/dev/<VG>/ubuntu-hvm,hda,w','file:/root/ubuntu-12.04-desktop-amd64.iso,hdc:cdrom,r'] vnc = 1 boot="dc"
xm create /etc/xen/ubuntu-hvm.cfg vncviewer localhost:0
After the install you can optionally remove the CDROM from the config and/or change the boot order.
For example /etc/xen/ubuntu-hvm.cfg:
builder = "hvm" name = "ubuntu-hvm" memory = "512" vcpus = 1 vif = [''] #disk = ['phy:/dev/<VG>/ubuntu-hvm,hda,w','file:/root/ubuntu-12.04-server-amd64.iso,hdc:cdrom,r'] disk = ['phy:/dev/<VG>/ubuntu-hvm,hda,w'] vnc = 1 boot="c" #boot="dc"
Xen Toolstack Choices
Xen and xl
xl is a new toolstack written from the ground up to be a replacement for xend and xm. Xen 4.1 contains a "tech preview" version of xl that is mostly functional, but may still contain some bugs and missing features. As of Xen 4.2, xl will have feature parity with xend, and will be the preferred toolstack. xend/xm are deprecated as of 4.2, and will be removed at some point.
To test xl, do the following:
sudo sed -i 's/TOOLSTACK=.*\+/TOOLSTACK="xl"/' /etc/default/xen sudo reboot sudo /etc/init.d/xend stop sudo xl list
xl and xm are very similar in functionality with a few notable exceptions: http://wiki.xen.org/wiki/XL
Xen and Libvirt
Make the following change to the xend configuration in /etc/xen/xend-config.sxp:
sudo /etc/init.d/xend restart
sudo apt-get install virtinst
sudo virt-install --name ubuntu --ram 256 --disk <path to LV or disk image> --location http://mirror.clarkson.edu/fedora/linux/releases/16/Fedora/x86_64/os/
Xen and XAPI
Other tips and tricks
Create and format disk image file
sudo mkdir -p /var/lib/xen/images sudo dd if=/dev/zero of=/var/lib/xen/images/ubuntu-guest.img bs=1M seek=3096 count=0 sudo mkfs.ext4 -F /var/lib/xen/images/ubuntu-guest.img
http://wiki.xen.org/wiki/Debian_Guest_Installation_Using_Debian_Installer - Netboot installation of PV guests
http://wiki.xen.org/wiki/HostConfiguration/Networking - Networking configuration details from Xen.org wiki
http://libvirt.org/uri.html#URI_file - Libvirt xend configuration
http://wiki.xen.org/wiki/Xen_Man_Pages - Xen Man pages
http://xenbits.xen.org/docs/unstable/man/xmdomain.cfg.5.html - xm config options
http://xenbits.xen.org/docs/unstable/man/xl.cfg.5.html xl config options
http://xenbits.xen.org/docs/unstable/misc/xl-disk-configuration.txt xl disk configuration
http://serverfault.com/questions/390373/xen-4-1-host-dom0-with-blktap-disks-tapaio-not-connecting blktap issues and fixes.