Ubuntu Release: JeOS 8.10 (Intrepid Ibex) - (Note : JeOS is discontinued as a standalone product. An equivalent installation can be done now with the standard Ubuntu Server installer by pressing F4 at the first screen)

Introduction

This page documents the building of a virtual appliance using Ubuntu Server Edition's JeOS and vmbuilder. It is an update of the original article, covering the same process, but using Ubuntu's vmbuilder.

However helpful this document may be for those people using vmbuilder to install a normal (but minimal) Ubuntu server, this document contains instructions specific to the building of an image intended to be mass-deployed (appliance).

What is JeOS?

Ubuntu JeOS (pronounced "Juice") is an efficient variant of the Ubuntu Server operating system, configured specifically for virtual appliances. It is no longer available as a CD image for download. Now you can install it with either:

• the Ubuntu Server Edition installation CD by pressing F4 and choosing "Install a minimal virtual machine", or
• Ubuntu's vmbuilder, as described here.

JeOS is a specialized installation of Ubuntu Server Edition with a tuned kernel that contains only the base elements needed to run within a virtualized environment.

Ubuntu JeOS has been tuned to take advantage of key performance technologies in the latest virtualization products from VMware. This combination of reduced size and optimized performance ensures that Ubuntu JeOS Edition delivers a highly efficient use of server resources in large virtual deployments.

Without unnecessary drivers, and only the minimal required packages, ISVs can configure their supporting OS exactly as they require. Updates, whether for security or enhancement reasons, will be limited to the bare minimum of what is required in their specific environment. In turn, users deploying virtual appliances built on top of JeOS will have to go through fewer updates and therefore less maintenance than they would have had to with a standard full installation of a server.

What is Ubuntu's vmbuilder?

vmbuilder is a script that automates the process of creating a ready to use VM based on Ubuntu. There is no need for a JeOS CD image. The currently supported hypervisors are KVM, Xen and VMware.

You can pass command line options to add extra packages, remove packages, choose which version of Ubuntu, which mirror, etc. On recent hardware with plenty of RAM, tmpdir in /dev/shm or using a tmpfs, and a local mirror, you can bootstrap a VM in less than a minute.

First introduced as a shell script in Ubuntu 8.04 LTS, ubuntu-vm-builder started as a hack to help developers test their new code in a virtual machine without having to restart from scratch each time. As Ubuntu administrators started to notice this script, a few of them went on improving it and adapting it for so many use cases that Søren Hansen (the author of the script and Ubuntu virtualization specialist, not the golf player) decided to rewrite it from scratch for Intrepid as a Python script with a few new design goals:

• develop it so that it can be reused by other distributions
• use a plug-in mechanism for all virtualization interactions so that others can easily add logic for other virtualization environments
• provide an easy to maintain web interface as an alternative to the command line interface

The general principles and commands remain the same.

Initial setup

In this article, we assume that you have installed and configured libvirt and KVM locally on the machine you are using. For details on how to perform this, please refer to:

• the development version documentation

• the KVM wiki page.

We also assume that you know how to use a text based text editor such as nano or vi. If you have not used any of them before, you can get an overview of the various text editors available by reading the PowerUsersTextEditors page. This tutorial has been done on KVM, but the general principle should remain on other virtualization technologies.

Install vmbuilder

The name of the package that we need to install is python-vm-builder:

• sudo apt-get install python-vm-builder

Note: if you are running hardy, you can still perform most of this using the older version of the package named ubuntu-vm-builder, there are only a few changes to the syntax of the tool for the command line parameters. On the other hand, all templates and configuration files syntax have been modified. Note as well that python-vm-builder includes a wrapper to the ubuntu-vm-builder command so that scripts written for hardy should remain functional on Intrepid (again to the exception of templates/configuration files).

Defining your virtual machine

Defining a virtual machine with Ubuntu's vmbuilder is quite simple, but here are a few things to consider:

• If you plan on shipping a virtual appliance, do not assume that the end user will know how to extend disk size to fit their need, so either plan for a large virtual disk to allow for your appliance to grow, or explain fairly well in your documentation how to allocate more space. It may be a good idea to store data on some separate external storage.
• Given that RAM is much easier to allocate in a VM, RAM size should be set to whatever you think is a safe minimum for your appliance.

The vmbuilder command line has 2 main parameters: the virtualization technology (hypervisor) and the targeted distribution. Optional parameters are quite numerous and can be found using the following command:

• vmbuilder <hypervisor> <distro> --help

Note: you need to specify which hypervisor and distribution you are targeting in order to get a full list of options as some of them are specific to the tools used.

In the following, we'll pick some of those parameters to define our virtual appliance.

Base parameters

As this example is based on KVM and Ubuntu 8.10 (Intrepid Ibex), and we are likely to rebuild the same virtual machine multiple times, we'll invoke vmbuilder with the following first parameters:

• sudo vmbuilder kvm ubuntu --suite intrepid --flavour virtual --arch i386  -o --libvirt qemu:///system

• --suite defines the Ubuntu release.

• --flavour specifies the virtual kernel (that's the one used to build a JeOS image).

• --arch i386  means a 32 bit machine.

• -o tells vmbuilder to overwrite the previous version of the VM.

• --libvirt tells the local virtualization environment to add the resulting VM to the list of available machines.

Notes:

• Because of the nature of operations performed by vmbuilder, it needs to have root privileges, hence the use of sudo.

• If your virtual machine needs to use more than 3 GiB of ram, you should build a 64 bit machine (--arch amd64).

• Before Ubuntu 8.10, the virtual kernel was only built for 32 bit architecture, so if you want to define an amd64 machine on Hardy, you should use --flavour server instead.

installation parameters

Configuring the network

Assigning a fixed IP address

As a virtual appliance may be deployed on various very different networks, it is very difficult to know what the actual network will look like. In order to simplify configuration, it is a good idea to take an approach similar to what network hardware vendors usually do, namely assigning an initial fixed IP address to the appliance in a private class network that you will provide in your documentation. An address in the range 192.168.0.0/24 is usually a good choice.

To do this we'll use the following parameters:

• --ip ADDRESS   IP address in dotted form (defaults to dhcp if not specified)
  --mask VALUE   IP mask in dotted form (default: 255.255.255.0)
  --net VALUE    IP net address (default: X.X.X.0)
  --bcast VALUE  IP broadcast (default: X.X.X.255)
  --gw ADDRESS   Gateway address (default: X.X.X.1)
  --dns ADDRESS  Name server address (default: X.X.X.1)

We assume for now that default values are good enough, so the resulting invocation becomes:

• sudo vmbuilder kvm ubuntu --suite intrepid --flavour virtual --arch i386 -o --libvirt qemu:///system --ip 192.168.0.100

Modifying the libvirt template to use bridging

Because our appliance will probably need to be accessed by remote hosts, we need to configure libvirt so that the appliance uses bridge networking. To do this we use vmbuilder's template mechanism to modify the default one.

In our working directory we create the template hierarchy and copy the template we want to modify:

• mkdir -p mytemplates/libvirt
  cp /etc/vmbuilder/libvirt/* mytemplates/libvirt/

we can then edit mytemplates/libvirt/libvirtxml.tmpl to change

•     <interface type='network'>
        <source network='default'/>
      </interface>

into

•     <interface type='bridge'>
        <source bridge='br0'/>
      </interface>

and specify on the command line where our modified templates are stored by adding the following option:

• --templates mytemplates

Notes:

• Templates will be automatically looked up in the following order at the following locations:
  1. /etc/vmbuilder/
  2. ~/.vmbuilder/

  3. path provided by the --templates option

• Templates use the very complete formatting syntax offered by Cheetah Template, which is fully described in its user guide.

Partitioning

Partitioning of the virtual appliance will have to take into consideration what you are planning to do with it. Because most appliances want to have separate storage for data, having a separate /var would make sense.

In order to do this vmbuilder provides us with --part:

• --part PATH
    Allows to specify a partition table in partfile each  line  of  partfile  should  specify
    (root first):
        mountpoint size
    where  size  is  in megabytes. You can have up to 4 virtual disks, a new disk starts on a
    line with '---'.  ie :
        root 1000
        /opt 1000
        swap 256
        ---
        /var 2000
        /log 1500

In our case we will define a text file name vmbuilder.partition which will contain the following:

• root 8000
  swap 4000
  ---
  /var 20000

Note that as we are using virtual disk images, the actual sizes that we put here are maximum sizes for these volumes.

Our command line now looks like:

• sudo vmbuilder kvm ubuntu --suite intrepid --flavour virtual --arch i386 -o --libvirt qemu:///system --ip 192.168.0.100 --part vmbuilder.partition --templates mytemplates

User and password

Again setting up a virtual appliance, you will need to provide a default user and password that is generic so that you can include it in your documentation. We will see later on in this tutorial how we will provide some security by defining a script that will be run the first time a user actually logs in the appliance, that will, among other things, ask him to change his password. In this example I will use 'user' as my user name, and 'default' as the password.

To do this we use the following optional parameters:

• --user USERNAME
    Sets the name of the user to be added. Default: ubuntu.
  --name FULLNAME
    Sets the full name of the user to be added. Default: Ubuntu.
  --pass PASSWORD
    Sets the password for the user. Default: ubuntu.

Our resulting command line becomes:

• sudo vmbuilder kvm ubuntu --suite intrepid --flavour virtual --arch i386 -o --libvirt qemu:///system --ip 192.168.0.100 --part vmbuilder.partition --user user --name user --pass default

Installing required packages

In this example we will be installing a package (Limesurvey) that accesses a mysql database and has a web interface. We will therefore require our OS to provide us with:

• Apache 2
• PHP
• MySQL
• OpenSSH server
• Limesurvey (as an example application that we have packaged)

This is done using vmbuilder by specifying the --addpkg command multiple times:

• --addpkg PKG
    Install PKG into the guest (can be specified multiple times)

However, due to the way vmbuilder operates, packages that have to ask questions to the user during the post install phase are not supported and should instead be installed while interactivity can occur. This is the case of Limesurvey, which we will have to install later, once the user logs in.

Other packages that ask simple debconf question, such as mysql-server asking to set a password, the package can be installed immediately, but we will have to reconfigure it the first time the user logs in.

If some packages that we need to install are not in main, we need to list all repositories that we want to use (including main) with --comp and --ppa:

• --components COMP1,COMP2,...,COMPN
             A comma-separated list of distribution components to include (e.g. main,universe). This defaults
             to "main"
  --ppa=PPA  Add ppa belonging to PPA to the VM's sources.list.

Limesurvey not being part of the archive at the moment, we'll specify it's PPA (personal package archive) address so that it is added to the VM /etc/apt/source.list, so we add the following options to the command line:

• --addpkg openssh-server --addpkg apache2 --addpkg apache2-mpm-prefork --addpkg apache2-utils --addpkg apache2.2-common --addpkg dbconfig-common --addpkg libapache2-mod-php5 --addpkg mysql-client --addpkg php5-cli --addpkg php5-gd --addpkg php5-ldap --addpkg php5-mysql --addpkg wwwconfig-common --addpkg mysql-server --ppa nijaba

OpenSSH

Another convenient tool that we want to have on our appliance is OpenSSH, as it will provide our admins to access to access the appliance remotely. However, pushing in the wild an appliance with a pre-installed OpenSSH host key is somewhat of a security risk, since it allows one server to masquerade as another. To fix this we'll use a --firstboot script to generate a new one. (Note that we can't reliably install the OpenSSH package at boot, since we are not guaranteed to have a working network by the time the boot script runs.)

Speed consideration

Packages caching

When vmbuilder creates builds your system, it has to go fetch each one of the packages that composes it over the network to one of the official repositories, which, depending on your internet connection speed and the load of the mirror, can have a big impact on the actual build time. In order to reduce this, it is recommended to either have a local repository (which can be created using apt-mirror) or using a caching proxy such as apt-cache. The later option being much simpler to implement and requiring less disk space, it is the one we will pick in this tutorial. To install it, simply type :

• sudo apt-get install apt-proxy

once this is complete, your (empty) proxy is ready for use on http://mirroraddress:9999 and will find ubuntu repository under /ubuntu. For vmbuilder to use it, we'll have to use the --mirror option

• --mirror=URL  Use Ubuntu mirror at URL instead of the default, which
                is http://archive.ubuntu.com/ubuntu for official
                arches and http://ports.ubuntu.com/ubuntu-ports
                otherwise

so we add to the command line:

• --mirror http://mirroraddress:9999/ubuntu

Note: the mirror address specified here will also be used in the /etc/apt/source.list of the newly created guest, so it is useful to specify here an address that can be resolved by the guest or to plan on resetting this address later on, such as in a --firstboot script. If you use an address which cannot be resolved, or use either 127.0.0.1 or localhost then any applications that you would like to install in your --firstlogin script, e.g. the application package, will not be installed. In that case, you may want to use the followings options that have been more recently added to vmbuilder:

•     --proxy=URL         Use proxy at URL for cached packages
      --install-mirror=URL
                          Use Ubuntu mirror at URL for the installation only.
                          Apt's sources.list will still use default or URL set
                          by --mirror Config option: install_mirror
      --security-mirror=URL
                          Use Ubuntu security mirror at URL instead of the
                          default, which is http://security.ubuntu.com/ubuntu
                          for official arches and http://ports.ubuntu.com
                          /ubuntu-ports otherwise. Config option:
                          security_mirror
      --install-security-mirror=URL
                          Use the security mirror at URL for installation only.
                          Apt's sources.list will still use default or URL set
                          by --security-mirror Config option:
                          install_security_mirror

Option: install a local mirror

If we are in a larger environment, it may make sense to setup a local mirror of the Ubuntu repositories. The package apt-mirror provides you with a script that will handle the mirroring for you. You should plan on having about 20 gigabyte of free space per supported release and architecture.

By default, apt-mirror uses the configuration file in /etc/apt/mirror.list. As it is set up, it will replicate only the architecture of the local machine. If you would like to support other architectures on your mirror, simply duplicate the lines starting with "deb", replacing the deb keyword by /deb-{arch} where arch can be i386, amd64, etc... For example, on an amd64 machine, to have the i386 archives as well, you will have :

• deb  http://archive.ubuntu.com/ubuntu hardy main restricted universe multiverse 
  /deb-i386  http://archive.ubuntu.com/ubuntu hardy main restricted universe multiverse 
  
  deb  http://archive.ubuntu.com/ubuntu hardy-updates main restricted universe multiverse 
  /deb-i386  http://archive.ubuntu.com/ubuntu hardy-updates main restricted universe multiverse 
  
  deb http://archive.ubuntu.com/ubuntu/ hardy-backports main restricted universe multiverse 
  /deb-i386  http://archive.ubuntu.com/ubuntu hardy-backports main restricted universe multiverse 
  
  deb http://security.ubuntu.com/ubuntu hardy-security main restricted universe multiverse 
  /deb-i386  http://security.ubuntu.com/ubuntu hardy-security main restricted universe multiverse 
  
  deb http://archive.ubuntu.com/ubuntu hardy main/debian-installer restricted/debian-installer universe/debian-installer multiverse/debian-installer 
  /deb-i386 http://archive.ubuntu.com/ubuntu hardy main/debian-installer restricted/debian-installer universe/debian-installer multiverse/debian-installer 

Notice that the source packages are not mirrored as they are seldom used compared to the binaries and they do take a lot more space, but they can be easily added to the list.

Once the mirror has finished replicating (and this can be quite long), you need to configure Apache so that your mirror files (in /var/spool/apt-mirror if you did not change the default), are published by your Apache server. I am not going to detail this process here, as Apache configuration is not our goal and many tutorials already exist on the subject.

Installing in a RAM disk

As you can easily imagine, writing to RAM is a LOT faster than writing to disk. If you have some free memory, letting vmbuilder perform its operation in a RAMdisk will help a lot and the option --tmpfs will help you do just that:

• --tmpfs OPTS  Use a tmpfs as the working directory, specifying its
                size or "-" to use tmpfs default (suid,dev,size=1G).

so adding

• --tmpfs -

sounds like a very good idea if you have 1G of free ram.

Packaging the application

Two option are available to us:

• The recommended method to do so is to make a Debian package. Since this is outside of the scope of this tutorial, we will not perform this here and invite the reader to read the documentation on how to do this at https://wiki.ubuntu.com/PackagingGuide. In this case it is also a good idea to setup a repository for your package so that updates can be conveniently pulled from it. See http://www.debian-administration.org/articles/286 for a tutorial on this.

• Manually install the application under /opt as recommended by the FHS guidelines at http://www.pathname.com/fhs/.

In our case we'll use Limesurvey as example web application for which we wish to provide a virtual appliance. As noted before, we've made a version of the package available in a PPA (personal package archive).

Finishing install on first boot and login

First boot

As we mentioned earlier, the first time the machine boots we'll need to reconfigure openssh-server so that the key generated for it is unique for each machine. Since we are planning to distribute our image, it would be really bad if all of our images had the same ssh host key as it would allow all kinds of attacks from anyone having full access to on of the images. We also use this place to expire our user account so that he is forced to change it the first time he logs in. To do this, we'll write a script called boot.sh as follow:

• # This script will run the first time the virtual machine boots
  # It is ran as root.
  
  # Expire the user account
  passwd -e user
  
  # Regenerate ssh keys
  rm /etc/ssh/ssh_host*key*
  dpkg-reconfigure -fnoninteractive -pcritical openssh-server

and we add the following option to our command line:

• --firstboot boot.sh

First login

Mysql and Limesurvey needing some user interaction during their setup, we'll set them up the first time a user logs in using a script named login.sh. We'll also use this script to let the user specify the keyboard and other locale info he wants to use:

• # This script is ran the first time a user logs in
  
  echo "Your appliance is about to be finished to be set up."
  echo "In order to do it, we'll need to ask you a few questions,"
  echo "starting by setting you keyboard and other console informations."
  
  #give the opportunity to change the keyboard
  sudo dpkg-reconfigure console-setup
  
  #configure the mysql server root password
  sudo dpkg-reconfigure mysql-server-5.0
  
  #install limesurvey
  sudo apt-get install -qqy --force-yes limesurvey
  
  echo "Your appliance is now configured.  To use it point your"
  echo "browser to http://serverip/limesurvey/admin"

and we add the following option to our command line:

• --firstlogin login.sh

Useful additions

Setting up for automated updates

To have your system be configured to update itself on a regular basis, we will just install unattended-upgrades, so we add the following option to our command line:

• --addpkg unattended-upgrades

As we have put our application package in a PPA, the process will update not only the system, but also the application each time we update the version in the PPA.

ACPI event handling

For your virtual machine to be able to handle restart and shutdown events it is being sent, it is a good idea to install the acpid package as well. To do this we just add the following option:

• --addpkg acpid

Final command line

• sudo vmbuilder kvm ubuntu --arch i386 --ip 192.168.0.100 --part vmbuilder.partition --user user --name user --pass default --tmpfs - --firstboot boot.sh --firstlogin login.sh --mirror http://mirroraddress:9999/ubuntu --suite intrepid --flavour virtual --addpkg openssh-server --addpkg apache2 --addpkg apache2-mpm-prefork --addpkg apache2-utils --addpkg apache2.2-common --addpkg dbconfig-common --addpkg libapache2-mod-php5 --addpkg mysql-client --addpkg php5-cli --addpkg php5-gd --addpkg php5-ldap --addpkg php5-mysql --addpkg wwwconfig-common --addpkg mysql-server --addpkg unattended-upgrades --addpkg acpid --ppa nijaba --libvirt qemu:///system 

Using configuration files

As you can see above, command lines can become quite long and it sometimes becomes useful to store some common parameters in a configurations file. If present, the /etc/vmbuilder.cfg and ~/.vmbuilder.cfg files can do just that, and you can also override them using the -c ALTCONFIG option, pointing to whatever other file you want.

The syntax of these files is as follow:

• [DEFAULT]
  <option> = <value>
  ...
  [<hypervisor>]
  <option> = <value>
  ...
  [<distro>]
  <option> = <value>
  ...
  [<hypervisor>/<distro>]
  <option> = <value>
  ...

where :

• <option>: is the name of the option as specified for the command line without -- (ie: user for --user). Python's ConfParser class is implemented so that it follows the following rules. --help include a "Conf name:" hint for those values that are ambiguous.

  • when an option has multiple form, the first encountered long name is the one to use in the conf file.

  • when an option contains - in its name, the - should be transformed in a _ in the conf file. (ie: in_place instead of in-place)

  • when the destination variable name is specified, it is the variable name that should be used (ie: destdir instead of dest).

• <value>: is the value to use for this option. Note that addpkg is a special case as you can add multiple values as addpkg = <package1>, <package2>, ..., <packageN>

• <hypervisor>: is the name of one of the valid hypervisor (ie: [kvm])

• <distro>: is the name of one of the valid distributions (ie: [ubuntu])

Values are evaluated in this order:

1. program default

2. content of the configuration files (/etc/vmbuilder.cfg, ~/.vmbuilder.cfg, file specified by -c), evaluated in that order. Sections are evaluated in the following order within each file:

   1. content of the [DEFAULT] section

   2. content of the appropriate [<hypervisor>] section

   3. content of the appropriate [<distro>] section

   4. content of the appropriate [<hypervisor>/<distro>] section

3. values specified on the command line

so that the last one encountered will always be the one that is used.

For example, we could simplify the above command line by using the following:

• sudo vmbuilder kvm ubuntu -c mylimesurveyappliance.cfg

where mylimesurveyappliance.cfg would contain:

• [DEFAULT]
  arch = i386
  ip = 192.168.0.100
  part = vmbuilder.partition
  user = user
  name = user
  pass = default
  tmpfs = - 
  firstboot = boot.sh
  firstlogin = login.sh 
  
  [ubuntu]
  mirror = http://mirroraddress:9999/ubuntu
  suite = intrepid 
  flavour = virtual
  addpkg = openssh-server, apache2, apache2-mpm-prefork, apache2-utils, apache2.2-common, dbconfig-common, libapache2-mod-php5, mysql-client, php5-cli, php5-gd, php5-ldap, php5-mysql, wwwconfig-common, mysql-server, unattended-upgrades, acpid
  ppa = nijaba 
  
  [kvm]
  libvirt = qemu:///system 

or, if mirror and tmpfs is something we always want for every VM, we can define them in ~/vmbuilder.cfg as:

• [DEFAULT]
  tmpfs = -
  
  [ubuntu]
  mirror = http://mirroraddress:9999/ubuntu

and remove the above 2 lines from the mylimesurveyappliance.cfg files.

Note: Configuration files are read using Python's ConfigParser module for more syntax details.

Conclusion

We sincerely hope that this tutorial will make building your own appliance simpler. For further documentation on the Ubuntu Server Edition, please refer to our Server Guide: https://help.ubuntu.com/8.10/serverguide/C/index.html. If you are interested in learning more, or have questions or suggestions, you are welcome to talk with us on:

• IRC: #ubuntu-server and #ubuntu-virt on Freenode

• the mailing list: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server

CategoryDoc