The libvirt library is used to interface with different virtualisation technologies. Before getting started with libvirt it is best to make sure your hardware supports the necessary virtualisation extensions for KVM. Enter the following from a terminal prompt:
A message will be printed informing you if your CPU does or does not support hardware virtualisation.
On many computers with processors supporting hardware assisted virtualization, it is necessary to activate an option in the BIOS to enable it.
There are a few different ways to allow a virtual machine access to the external network. The default virtual network configuration includes bridging and iptables rules implementing usermode networking, which uses the SLIRP protocol. Traffic is NATed through the host interface to the outside network.
To enable external hosts to directly access services on virtual machines a different type of bridge than the default needs to be configured. This allows the virtual interfaces to connect to the outside network through the physical interface, making them appear as normal hosts to the rest of the network.
To install the necessary packages, from a terminal prompt enter:
sudo apt install qemu-kvm libvirt-bin
After installing libvirt-bin, the user used to manage virtual machines will need to be added to the libvirtd group. Doing so will grant the user access to the advanced networking options.
In a terminal enter:
sudo adduser $USER libvirtd
If the user chosen is the current user, you will need to log out and back in for the new group membership to take effect.
In more recent releases (>= Yakkety) the group was renamed to libvirt. Upgraded systems get a new libvirt group with the same gid as the libvirtd group to match that.
You are now ready to install a Guest operating system. Installing a virtual machine follows the same process as installing the operating system directly on the hardware. You either need a way to automate the installation, or a keyboard and monitor will need to be attached to the physical machine.
In the case of virtual machines a Graphical User Interface (GUI) is analogous to using a physical keyboard and mouse. Instead of installing a GUI the virt-viewer application can be used to connect to a virtual machine's console using VNC. See Virtual Machine Viewer for more information.
There are several ways to automate the Ubuntu installation process, for example using preseeds, kickstart, etc. Refer to the Ubuntu Installation Guide for details.
Yet another way to install an Ubuntu virtual machine is to use uvtool. This application, available as of 14.04, allows you to set up specific VM options, execute custom post-install scripts, etc. For details see Cloud images and uvtool.
Libvirt can also be configured work with Xen. For details, see the Xen Ubuntu community page referenced below.
virt-install is part of the virtinst package. To install it, from a terminal prompt enter:
sudo apt install virtinst
There are several options available when using virt-install. For example:
sudo virt-install -n web_devel -r 512 \ --disk path=/var/lib/libvirt/images/web_devel.img,bus=virtio,size=4 -c \ ubuntu-18.04-server-i386.iso --network network=default,model=virtio \ --graphics vnc,listen=0.0.0.0 --noautoconsole -v
-n web_devel: the name of the new virtual machine will be web_devel in this example.
-r 512: specifies the amount of memory the virtual machine will use in megabytes.
--disk path=/var/lib/libvirt/images/web_devel.img,size=4: indicates the path to the virtual disk which can be a file, partition, or logical volume. In this example a file named web_devel.img in the /var/lib/libvirt/images/ directory, with a size of 4 gigabytes, and using virtio for the disk bus.
-c ubuntu-18.04-server-i386.iso: file to be used as a virtual CDROM. The file can be either an ISO file or the path to the host's CDROM device.
--network provides details related to the VM's network interface. Here the default network is used, and the interface model is configured for virtio.
--graphics vnc,listen=0.0.0.0: exports the guest's virtual console using VNC and on all host interfaces. Typically servers have no GUI, so another GUI based computer on the Local Area Network (LAN) can connect via VNC to complete the installation.
--noautoconsole: will not automatically connect to the virtual machine's console.
-v: creates a fully virtualised guest.
After launching virt-install you can connect to the virtual machine's console either locally using a GUI (if your server has a GUI), or via a remote VNC client from a GUI-based computer.
The virt-clone application can be used to copy one virtual machine to another. For example:
sudo virt-clone -o web_devel -n database_devel -f /path/to/database_devel.img
-o: original virtual machine.
-n: name of the new virtual machine.
-f: path to the file, logical volume, or partition to be used by the new virtual machine.
Also, use -d or --debug option to help troubleshoot problems with virt-clone.
Replace web_devel and database_devel with appropriate virtual machine names.
Virtual Machine Management
There are several utilities available to manage virtual machines and libvirt. The virsh utility can be used from the command line. Some examples:
To list running virtual machines:
To start a virtual machine:
virsh start web_devel
Similarly, to start a virtual machine at boot:
virsh autostart web_devel
Reboot a virtual machine with:
virsh reboot web_devel
The state of virtual machines can be saved to a file in order to be restored later. The following will save the virtual machine state into a file named according to the date:
virsh save web_devel web_devel-022708.state
Once saved the virtual machine will no longer be running.
A saved virtual machine can be restored using:
virsh restore web_devel-022708.state
To shutdown a virtual machine do:
virsh shutdown web_devel
A CDROM device can be mounted in a virtual machine by entering:
virsh attach-disk web_devel /dev/cdrom /media/cdrom
In the above examples replace web_devel with the appropriate virtual machine name, and web_devel-022708.state with a descriptive file name.
If virsh (or other vir* tools) shall connect to something else than the default qemu-kvm/system hipervisor one can find alternatives for the connect option in man virsh or libvirt doc
There are different types of migration available depending on the versions of libvirt and the hipervisor being used. In general those types are:
There are various options to those methods, but the entry point for all of them is virsh migrate. Read the integrated help for more detail.
virsh migrate --help
Some useful documentation on constraints and considerations about live migration can be found at the Ubuntu Wiki
Device Passthrough / Hotplug
If instead of the here described hotplugging you want to always pass through a device add the xml content of the device to your static guest xml representation via e.g. virsh edit <guestname>. In that case you don't need to use attach/detach. There are different kinds of passthrough. Types available to you depend on your Hardware and software setup.
But both kinds are handled in a very similar way and while there are various way to do it (e.g. also via qemu monitor) driving such a change via libvirt is recommended. That way libvirt can try to manage all sorts of special cases for you and also somewhat masks version differences.
In general when driving hotplug via libvirt you create a xml snippet that describes the device just as you would do in a static guest description. A usb device is usually identified by Vendor/Product id's:
<hostdev mode='subsystem' type='usb' managed='yes'> <source> <vendor id='0x0b6d'/> <product id='0x3880'/> </source> </hostdev>
<hostdev mode='subsystem' type='pci' managed='yes'> <source> <address domain='0x0000' bus='0x04' slot='0x10' function='0x0'/> </source> </hostdev>
To get the Virtual function in the first place is very device dependent and can therefore not be fully covered here. But in general it involves setting up an iommu, registering via VFIO and sometimes requesting a number of VFs. Here an example on ppc64el to get 4 VFs on a device:
$ sudo modprobe vfio-pci # identify device $ lspci -n -s 0005:01:01.3 0005:01:01.3 0200: 10df:e228 (rev 10) # register and request VFs $ echo 10df e228 | sudo tee /sys/bus/pci/drivers/vfio-pci/new_id $ echo 4 | sudo tee /sys/bus/pci/devices/0005\:01\:00.0/sriov_numvfs
You then attach or detach the device via libvirt by relating the guest with the xml snippet.
virsh attach-device <guestname> <device-xml> # Use the Device int the Guest virsh detach-device <guestname> <device-xml>
There are several associated known issues in regard to apparmor proetction protecting "too much". You might need to tweak exceptions in the apparmor profiles until the bugs 1552241 (for USB) and https://bugs.launchpad.net/ubuntu/+source/apparmor/+bug/1679704 (For VF assignment) are resolved. To check if you are affected watch dmesg while you use the USB/VF passthrough/hotplug feature and verify if you see apparmor denies.
Access Qemu Monitor via libvirt
The Qemu Monitor is the way to interact with qemu/KVM while a guest is running. This interface has many and very powerful features for experienced users. When running under libvirt that monitor interface is bound by libvirt itself for management purposes, but a user can run qemu monitor commands via libvirt still. The general syntax is virsh qemu-monitor-command [options] [guest] 'command'
Libvirt covers most use cases needed, but if you every want/need to work around libvirt or want to tweak very special options you can e.g. add a device that way:
virsh qemu-monitor-command --hmp zesty-test-log 'drive_add 0 if=none,file=/var/lib/libvirt/images/test.img,format=raw,id=disk1'
But since the monitor is so powerful, you can do a lot especially for debugging purposes like showing the guest registers:
virsh qemu-monitor-command --hmp y-ipns 'info registers' RAX=00ffffc000000000 RBX=ffff8f0f5d5c7e48 RCX=0000000000000000 RDX=ffffea00007571c0 RSI=0000000000000000 RDI=ffff8f0fdd5c7e48 RBP=ffff8f0f5d5c7e18 RSP=ffff8f0f5d5c7df8 [...]
Virtual Machine Manager
The virt-manager package contains a graphical utility to manage local and remote virtual machines. To install virt-manager enter:
sudo apt install virt-manager
Since virt-manager requires a Graphical User Interface (GUI) environment it is recommended to be installed on a workstation or test machine instead of a production server. To connect to the local libvirt service enter:
virt-manager -c qemu:///system
You can connect to the libvirt service running on another host by entering the following in a terminal prompt:
virt-manager -c qemu+ssh://virtnode1.mydomain.com/system
The above example assumes that SSH connectivity between the management system and virtnode1.mydomain.com has already been configured, and uses SSH keys for authentication. SSH keys are needed because libvirt sends the password prompt to another process. For details on configuring SSH see OpenSSH Server
Virtual Machine Viewer
The virt-viewer application allows you to connect to a virtual machine's console. virt-viewer does require a Graphical User Interface (GUI) to interface with the virtual machine.
To install virt-viewer from a terminal enter:
sudo apt install virt-viewer
Once a virtual machine is installed and running you can connect to the virtual machine's console by using:
Similar to virt-manager, virt-viewer can connect to a remote host using SSH with key authentication, as well:
virt-viewer -c qemu+ssh://virtnode1.mydomain.com/system web_devel
Be sure to replace web_devel with the appropriate virtual machine name.
If configured to use a bridged network interface you can also setup SSH access to the virtual machine.
See the KVM home page for more details.
For more information on libvirt see the libvirt home page
The Virtual Machine Manager site has more information on virt-manager development.
Also, stop by the #ubuntu-virt IRC channel on freenode to discuss virtualisation technology in Ubuntu.
Another good resource is the Ubuntu Wiki KVM page.
For information on Xen, including using Xen with libvirt, please see the Ubuntu Wiki Xen page.
For basics how to assign VT-d devices to qemu/KVM, please see the linux-kvm page.