How to fix missing Virtual Machines after upgrading from 10.04 (Lucid) to 12.04 (Precise)?

Are all your KVM virtual machines missing after upgrading between these two LTS releases? If so, check Ubuntu Bug #931350 for more information and how to fix the problem. Specifically, you can fix this issue by downloading and running the simple script provided by Peter Matulis in Comment #9.

How to prevent VirtualBox and KVM from conflicting?

Installing KVM loads the module kvm (or kvm-intel in my case) which does not allow other virtualization software such as VirtualBox to run.

The good news is that you can disable KVM and unload its modules:

1. Make sure you've completely shut down and powered off all your KVM virtual machines (using virt-manager or otherwise).

2. Unload all of its modules:

$ sudo invoke-rc.d kvm stop

In Karmic use

$ sudo invoke-rc.d qemu-kvm stop

instead.

In Lucid and later, use

$ sudo service qemu-kvm stop

instead.

How to boot a VM into single mode?

A VM has a complete BIOS, and also runs GRUB. Unfortunately, the GRUB menu disappears after a couple of seconds, which often doesn't leave time to connect using virt-viewer. There is a small trick here: shutdown the VM, run virt-viewer with the argument --wait (that way it will wait for the VM to start and pop up directly after that), and then start the VM; this should give you a good 2 seconds time to enter the BIOS or access the GRUB menu.

If you are running a VM built with ubuntu-vm-builder, there is a small bug that will prevent you from logging in; in GRUB, select the recovery line and press "e"; select the line beginning with kernel, press "e" again. At the end of the line, add "init=/bin/sh" (without the quotes). Confirm with enter, and press b to boot the kernel with these settings.

How to convert VMware machines to virt-manager?

If you want to convert a Windows (XP) vmdk, you must first do two things while running the VM in VMware:

  1. Enable IDE in the registry. Start the machine and execute mergeide.reg. See the ProxMox wiki for very detailed instructions.

  2. Remove VMware tools via the Control Panel.

If the converted VM still displays a BSOD on boot, you can resolve the issue as follows by booting the converted VM booting from a Windows XP CD and choosing the 'Install' options (not 'R = Repair'). After a while, Windows setup should display the existing Windows installation (located in C:\Windows by default) and offer the option to repair that installation. Windows Updates will be reset after this procedure, but all your programs, settings, documents, etc. will be preserved. After this procedure, you will need to reactivate Windows, but the VM will boot.

To use the VMware machine from within virt-manager, the .vmx file must be converted to libvirt's .xml. vmware2libvirt was created to help with this. It's available in Intrepid/Jaunty, install package 'virt-goodies'. After installing it can be used like so:

$ vmware2libvirt -f ./file.vmx > file.xml
$ virsh -c qemu:///system define file.xml

The first command converts the VMware 'file.vmx' file to the libvirt compatible 'file.xml'. See  man vmware2libvirt  for details. The second command imports file.xml into libvirt. The imported .xml files are stored in /etc/libvirt/qemu.

Caveats

While vmware2libvirt works well on simple virtual machines, there are limitations because .vmx files don't always contain enough information, and also because vmware2libvirt tries not to make too many assumptions about what it finds. A couple of things to look out for:

  1. While vmware2libvirt attempts to detect a 64-bit guest, be sure that your 64-bit guest has in its .xml file:

    <os>
     <type arch='x86_64' machine='pc'>hvm</type>
     ...
    </os>
  2. vmware2libvirt only detects and uses the first Ethernet interface found. Additional interfaces can be added from within virt-manager.
  3. Currently the first scsi disk is used if found, otherwise the first ide disk. Additional disks can be added from within virt-manager.
  4. The converted virtual machine is hard-coded to use 1 CPU. This can be changed with:

    <vcpu>2</vcpu>
  5. vmware2libvirt does not (and cannot) convert anything that was VMware-specific within the guest. See 'Guest Notes' below for more details.

Using and Converting VMware Virtual Disk Files

KVM has the ability to use VMware's .vmdk disk files directly, as long as the disk is wholly contained in a single vmdk file. However, VMware also allows splitting a disk into multiple smaller vmdk files, usually 2 GB each. KVM can't use these. You can convert these files into a single virtual disk file using vmware-vdiskmanager, included in the freely available VMware Server.

$ vmware-vdiskmanager -r <Name of split vmdk base file> -t 0 <Name of new single vmdk file>

IMPORTANT: before converting the disk files into a single disk file, you must first remove all snapshots (through the web interface if you were using VMware Server 2.0). Otherwise your merged disk will not be in the latest known state. Removing these snapshots can take quite some time, after which *-000001.vmdk - type of files may have disappeared from the file system.

To create a new virtual machine from an existing file, call virt-install with the --import argument (see the virt-install manpage for details).

To use a .vmdk disk in an existing virtual machine, modify the VM's XML file in /etc/libvirt/qemu:

 ...
 <disk type='file' device='disk'>
      <driver name='qemu' type='vmdk'/>
      <source file='/var/lib/libvirt/images/diskname.vmdk'/>
      <target dev='hda' bus='ide'/>
 </disk>
 ...

and redefine it:

$ virsh -c qemu:///system define NameOfMachine.xml

IMPORTANT: keep in mind that while the .vmx file is converted to .xml, the disks are used as-is. Please make backups, especially if you want to use the virtual machine in VMware later.

KVM is not able to make snapshots when using vmdk disk files, so converting the virtual disk file in qemu's qcow2 format is advisable. The qemu package contains a utility named qemu-img to do this:

qemu-img convert diskname.vmdk -O qcow2 diskname.qcow2
OR for raw
qemu-img convert diskname.vmdk -O raw diskname.raw

The converted image can be used as part of the definition of a new VM, or incorporated into an existing VM (see above). Adding the disk through virt-manager will always add it as a raw disk, so you will need to edit the xml as follows below to make sure it works with qcow2. The important bit is type='qcow2' instead of type='raw'!

...
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/.../diskname.qcow2'/>
      <target dev='hdb' bus='ide'/>
    </disk>
...

Should VMware Tools be kept after conversion?

If converting from vmware to libvirt, be sure to remove vmware-tools if you have it installed (otherwise it will overwrite xorg.conf on reboot)

How to convert Physical Machines to virt-manager

If you want to convert a Windows (XP) physical machine, you may need to enable IDE in the registry first. Start the machine and execute mergeide.reg. See the ProxMox wiki for very detailed instructions.

Using VMware Converter

VMware Converter is a free tool to migrate a physical machine to VMware. So the first step is to migrate the physical machine to a VMware image. The second step is to follow the howto "How to convert VMware Machines to virt-manager" (see above).

Using Linux tools

See the german book about qemu physical to virtual. There is a link to translate it in English.

How to convert VirtualBox machines to virt-manager?

First we need to convert the disk image format of all disks.

VBoxManage clonehd disk.vdi disk.raw --format raw
qemu-img convert -f raw disk.raw -O qcow2 disk.qcow2

Do this for all disks.

Then, unfortunately, you need to create a new KVM VM with the same disks. If it's a linux box, be sure to check first /etc/fstab to see which device names are used to mount the disks (sda, hda, vda). Check (using virsh edit) the VM's xml to make sure the same device names are used in the new KVM VM. Or edit the VM to use the new device names.

Should NTP be used for time synchronisation?

NTP can be used on guest VMs for time synchronisation. In particular, the KVM wiki recommends specifying the KVM host as the main NTP server within the VM instead of external NTP servers. This helps maintain tight coupling between guests and host. In addition, this thread also recommends using NTP in the guest VM to maintain proper timing. Note, it is not necessary to use NTP in a guest VM, as the VM still has access to the kvm-clock clocksource which will set the guest VMs hardware clock. More information about hardware clocks can be found here.

Which driver should be used for Xorg?

Video

When you first install a Linux guest, use the "cirrus" video driver. Post installation, if you wish to have a resolution above 800x600, switch to the standard video driver.

xorg.conf is "depreciated" and so is by default missing in Linux. To use the "standard" video driver use the following in /etc/X11/xorg.conf

Section "ServerLayout"
        Identifier     "BodhiZazen's KVM xorg.conf"
        Screen      0  "Screen0" 0 0
        InputDevice    "Mouse0" "CorePointer"
        InputDevice    "Keyboard0" "CoreKeyboard"
EndSection

Section "Module"
        Load  "record"
        Load  "dri"
        Load  "extmod"
        Load  "glx"
        Load  "dbe"
        Load  "dri2"
EndSection

Section "InputDevice"
        Identifier  "Keyboard0"
        Driver      "kbd"
EndSection

Section "InputDevice"
        Identifier  "Mouse0"
        Driver      "vmmouse"
        Option      "Protocol" "SysMouse"
        Option      "Device" "/dev/sysmouse"
        Option      "ZAxisMapping" "4 5 6 7"
EndSection

Section "Monitor"
        Identifier   "Monitor0"
        VendorName   "Monitor Vendor"
        ModelName    "Monitor Model"
        HorizSync       20.0 - 50.0
        VertRefresh     40.0 - 80.0
        Option          "DPMS"

EndSection

Section "Device"
        Identifier  "Card0"
        Driver      "vesa"
        VendorName  "KVM - std"
        BoardName   "GD 5446"
        BusID       "PCI:0:2:0"
EndSection

Section "Screen"
        Identifier "Screen0"
        Device     "Card0"
        Monitor    "Monitor0"
        SubSection "Display"
                Viewport   0 0
                Modes "1600x1200"
        EndSubSection
EndSection

Change the line "Modes "1600x1200" " to your desired resolution (I have been able to achieve resolutions up to 1920x1200 ). Hint - 1400X1050 is nice a well.

Notice the mouse driver is vmmouse . vmmouse is included by default in most distros (Debian, Fedora, and Ubuntu all include this driver by default).

Setting the standard video driver varies by how you use KVM.

Command line - use the option -vgs std

kvm -hda ubuntu.qcow -vga std

Virtmanager (graphical front end}

With the guest shut down, go to the guest over view page , Select the "Video" tab on the Right, select "vga" (rather then cirrus) from the pull down menu.

Libvirt - Edit your guest xml file, set the video to std with :

<model type='vga' vram='9216' heads='1'/>

If the display fails to start with an error in the logs like: "No core pointer", then the follow entry in the xorg.conf file will get around this (CentOS5 does this):

Section "ServerFlags"
        Option   "AllowMouseOpenFail" "1"
EndSection

How to set up guest resolution in Xorg?

Assuming you are using the xorg.conf from above you can set your resolution either by editing /etc/X11/xorg.conf and setting a new default resolutino (the Modes line as above) or, with the guest running, use the graphical system menu to set a Resolution.

In gnome this is System -> Preferences -> Monitors

In XFCE this is Applications -> Settings -> Display

In LXDE this is Menu -> Prefences -> Monitor Settings

What system specific adjustments are recommended?

Windows (<Vista)

Windows (other than Vista) virtual machines should substitute in the .xml file:

<features>
  <acpi/>
</features>

with:

<features/>

Windows Vista

Windows Vista virtual machines should add this to the xml file:

<domain type='kvm'>
...
  <features>
    <acpi/>
  </features>
</domain>

Debian Sarge

Debian Sarge (oldstable) may write /boot/grub/menu.lst incorrectly, resulting in the following on boot:

pivot_root: no such file or directory
/sbin/init: 432: cannot open dev/console: no such file
Kernel panic: Attempted to kill init!

The fix is at the grub menu, press 'e' and adjust root=/dev/hdb1 to be root=/dev/hda1. After a successful boot, you will need to update /boot/grub/menu.lst to have:

# kopt=root=/dev/hda1 ro

and run:

# update-grub

OpenBSD 4.1 (and others?)

OpenBSD 4.1 (and probably others) when using the rtl8139 network driver may realize poor network performance and see this message on the console:

re0: watchdog timeout

The fix is to use another NIC (e1000 appears to work well). See 'Changing the Network Card Model' above.

OpenBSD 4.5 and later

A change in the OpenBSD kernel causes it to hang on boot at "setting tty flags". If this affects you, follow the instructions on http://scie.nti.st/2009/10/4/running-openbsd-4-5-in-kvm-on-ubuntu-linux-9-04

Nested usage

Nested KVM (running kvm inside a kvm vm) tends to work pretty well for development purposes. However, through the releases it has had many regressions, and upstream does not consider it a supportable use case. Therefore it is not recommended for "real" workloads.

When using nested kvm, it is recommended to (a) use hugepages to back the parent VM, and (b) ensure that KSM is disabled in the nested VMs.


CategoryVirtualization

KVM/FAQ (last edited 2015-06-18 16:29:38 by 50-193-64-142-static)