UEC/Images

Contents

Intro
Daily and Released Images
Features of the Images
Machine Consumable Data [query]

Intro

The term 'UEC Images' refers to the Official Ubuntu images that are available at http://uec-images.ubuntu.com . These images are built by Canonical. They are then registered on EC2, and compressed tarfiles are made available for download. The downloaded content can be used as a starting point for creating your own image, or published unmodified to your own UEC.

Daily and Released Images

"Daily" builds are made available for testing and continuous integration. The daily builds are not tested at all. However, the content of the images has already been through the SRU process, so in general the daily builds for stable releases are stable. Quality of daily builds for the development release will vary.

Prior to a milestone or release, a daily builds is selected, tested and then promoted. Only a limited number of daily images are kept around (see the removal policy). Any build that is marked as 'released' will never be deleted.

Daily builds are available under http://uec-images.ubuntu.com/server/<codename> . For example, daily builds of lucid (10.04 LTS) are available at http://uec-images.ubuntu.com/server/lucid/ . Released builds are available under http://uec-images.ubuntu.com/server/releases/<codename> .

To use the images on EC2, simply start an instance with the command line tools using command lines given at uec-images, or launch one of these from the Amazon EC2 Console.

Features of the Images

Version Controlled Build Process

Its not really an end user feature, but a reliable build process in place. Images are built on a 10.04 LTS host from a 0.11 branch of vmbuilder. The build scripts are available under the automated-ec2-builds, and published to uec-images and EC2 using ec2-publishing-scripts.

CloudInit

The images contain CloudInit. CloudInit allows for dynamic run-time customization of images via user-data.

UnCloudInit

The images contain uncloud-init, a tool that helps run Ubuntu cloud images in a local hypervisor. uncloud-init helps set a password for user "ubuntu", enables password authentication in sshd configuration and spawns a login tty on the serial console

Run Images locally without a Cloud

Images after 10.10 can now be run outside of the cloud environment without any difficult setup, and even provide an easy way to provide metadata to the instance.

Download a suitable image from http://uec-images.ubuntu.com/server/maverick/current/ . The file name will vary. For this example, it is "maverick-server-uec-amd64.tar.gz".
```
$ tarball=maverick-server-uec-amd64.tar.gz
```

Extract the tar file and set some variables names for its contents. Here is an example

$ contents=${tarball}.contents
$ tar -Sxvzf ${tarball} | tee "${contents}"
maverick-server-uec-amd64.img
maverick-server-uec-amd64-vmlinuz-virtual
maverick-server-uec-amd64-loader
maverick-server-uec-amd64-floppy
README.files
$ base=$(sed -n 's/.img$//p' "${contents}")
$ kernel=$(echo ${base}-vmlinuz-*)
$ floppy=${base}-floppy
$ img=${base}.img

Create a copy on write disk image, based on the disk image we downloaded and boot that image in local kvm

# make a qcow image just to keep the original pristine
$ qemu-img create -f qcow2 -b ${img} disk.img
$ kvm -fda ${floppy} -drive if=virtio,file=disk.img -boot a

Get the password for 'ubuntu' and log in
The default selection in the grub menu will result in a randomly generated password for the 'ubuntu' user on first boot. The password is written two places, the console and the serial device. On the console, you will see lines like:

[    1.820549] EXT4-fs (sda): re-mounted. Opts: (null)
::uncloud-init: settting ubuntu pass = qlnYZTSy
::uncloud-init: changing ubuntu user's password!
::uncloud-init: enabled console on ttyS0

The password is also written to the kvm serial console. You can see the kvm serial console by hitting ctrl-alt-3. On first boot, you will see lines like

===                                                                             
ubuntu_pass = PhAKp1kx                                                          
===

Want to get that first boot again? Easy, just delete the cow disk, create a new one and boot it

$ rm -rf disk.img && qemu-img create -f qcow2 -b ${img} disk.img
$ kvm -fda ${floppy} -drive if=virtio,file=disk.img -boot a

Advanced use of Images

Command line parameters used by 'cloud-init' and 'uncloud-init'
By modifying the kernel command line, you can change the behavior of the 2 programs affect the early boot.
- ubuntu-pass=[password|random]
  On first boot, uncloud-init will set the password to be 'password'. If the string provided is literally the string 'random' then a random pass will be created and written to the console.
- ds=nocloud;[key=val] or ds=nocloud-net;[key=val]
  ds=nocloud
  Means the instance is booting on a plain hypervisor with no cloud metadata service to be expected
  ds=nocloud-net
  Means the instance is booting on a cloud of some sort, however do not expect the cloud metadata service to be available! This helps cloud-init not wait for the metadata service to appear. You can pass a custom metadata network service by passing the s= parameter
  - s=<seedlocation> | seedfrom=<seedlocation> [default=""]
    This allows you to set the 'seed location'. The value can be a url of 'file:///' format, or of 'http://' format. If you want to use http or another network based seed, then 'nocloud-net' must be used as the data source, rather than 'nocloud'. meta-data and user-data will be read from <seedlocation>meta-data and <seedlocation>user-data respectively.
    For example I set up the tiny urls http://tinyurl.com/sm-user-data and http://tinyurl.com/sm-meta-data . These redirect to pages on my hosted system, the tinyurl is only used to make the urls shorter. Then, I can boot with a command line like:
```
 "ro init=/usr/lib/cloud-init/uncloud-init root=LABEL=uec-rootfs ds=nocloud-net;s=http://tinyurl.com/sm- ubuntu-pass=random"
```
    in , and the user-data and meta-data are pulled from the respective URLs. The format of user-data and meta-data is described in cloud-init's documentation doc/examples/seed.
  - h=hostname | local-hostname=hostname [default="ubuntuhost"]
    Set the local hostname for this instance.
  - i=id | instance-id=id [default="nocloud"]
    This sets the the 'instance-id' for the boot. In EC2 or Eucalyptus, this would be something like 'i-abcdefg' and changes for each instance. The cloud-init code uses 'instance-id' to indicate whether this is the first boot. What this means is that, by default, the constant string 'nocloud' as an instance id will on ever initialize once. If youwant to test initialization again, you would have to remove files in /var/lib/cloud/sem or change this string.

boot with kvm '-kernel' command line arguments rather than with the boot floppy:

kvm -drive file=disk.img,if=virtio,boot=on \
   -kernel "${kernel}" \
   -append "root=/dev/vda ro init=/usr/lib/cloud-init/uncloud-init ds=nocloud ubuntu-pass=ubuntu"

Import the virtual machine into libvirt (replace PATHTO). Import the following xml with

virsh "create ./maverick-server-uec-i386.xml"

<domain type='kvm'>
  <name>maverick-server-uec-i386</name>
  <memory>131072</memory>
  <currentMemory>131072</currentMemory>
  <vcpu>1</vcpu>
  <os>
    <type arch='i686' machine='pc'>hvm</type>
    <boot dev='hd'/>
    <kernel>/PATHTO/maverick-server-uec-i386-vmlinuz-virtual</kernel>
    <cmdline>root=/dev/vda ro init=/usr/lib/cloud-init/uncloud-init ds=nocloud ubuntu-pass=ubuntu</cmdline>
  </os>
  <features>
    <acpi/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/bin/kvm</emulator>
    <disk type='file' device='disk'>
      <source file='/PATHTO/disk.img'/>
      <target dev='vda' bus='virtio'/>
    </disk>
    <interface type='network'>
      <source network='default'/>
    </interface>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes' listen='127.0.0.1'/>
    <video>
      <model type='cirrus' vram='9216' heads='1'/>
    </video>
  </devices>
</domain>

By providing user-data and meta-data via the seed, you can avoid the need for uncloud-init. Note that in this case, you will not be able to login to the instance over the console:
```
kvm -drive file=disk.img,if=virtio,boot=on \
   -kernel "${kernel}" \
   -net nic,model=virtio -net "user,hostfwd=tcp::5555-:22" \
   -append "root=/dev/vda ro ds=nocloud-net;s=http://tinyurl.com/sm-" \
   -curses
```
Given the above, which sets up redirections on localhost port 5555 to the guest's port 22, and suitable data in the referenced metadata and user-data, I can then ssh into the system with
```
ssh -p 5555 ubuntu@localhost
```

Use uncloud's "xupdate" mechanism to seed or update the filesystem:

# create a directory tree for var/lib/cloud/data/cache/nocloud
$ sdir="var/lib/cloud/data/cache/nocloud"
$ ud=overlay
$ mkdir -p "${ud}/updates/${sdir}"

# populate the nocloud's cache dir with some meta-data and user-data
$ b="http://bazaar.launchpad.net/%7Ecloud-init-dev/cloud-init/trunk/download/head%3A/preseedmetadata.txt-20100811204932-d0e2s9pcebnirj8p-1/"
$ wget "${b}/meta-data" -O "${ud}/updates/${sdir}/meta-data"
$ wget "${b}/user-data" -O "${ud}/updates/${sdir}/user-data"

# the 'updates.script' in the top level of this iso will
# be executed immediately after the contents of the updates/
# dir are synced to /
$ cat > "${ud}/updates.script" <<EOF
#!/bin/sh
echo ==== HELLO WORLD =====
sleep 3
EOF
$ chmod 755 "${ud}/updates.script"

# create a filesystem to house the data
$ genisoimage -rock --output updates.iso "${ud}"

# clean the disk image that we're going to boot
$ qemu-img create -f qcow2 -b ${img} disk.img

# launch kvm with the updates.iso as a virtio device
# and pass in 'xupdate=vdb:mnt', indicating that updates are
# on /dev/vdb, and the device contains a filesystem.
$ kvm -kernel "${kernel}" \
   -drive file=disk.img,if=virtio -drive file=updates.iso,if=virtio \
   -append "root=/dev/vda ro init=/usr/lib/cloud-init/uncloud-init ds=nocloud ubuntu-pass=ubuntu xupdate=vdb:mnt" \
   -curses -net nic,model=virtio -net "user,hostfwd=tcp::5555-:22"

Machine Consumable Data [query]

In order to provide information about what builds are available for download or running on ec2, a 'query' interface is exposed at http://uec-images.ubuntu.com/query . This will allow users of the service to download images or find out the latest ec2 AMIs programmatically.

The data is laid out as follows:

There are 2 files in top level director 'daily.latest.txt' and 'released.latest.txt'. Each of these files contains tab delimited data, with 4 fields per record. daily.latest.txt has information about the daily builds, released.latest.tt about released builds:
```
   <suite> <build_name> <label>     <serial>
   hardy   server       release     20100128
```

For each record in the top level files another set of files will exist: \

File	Description
<suite>/<build_name>/daily-dl.txt	downloadable images data for all daily builds currently available
<suite>/<build_name>/daily-dl.current.txt	downloadable images data for the most recent daily build
<suite>/<build_name>/released-dl.txt	downloadable images data for all released builds
<suite>/<build_name>/released-dl.current.txt	downloadable images data for the most recent released build
<suite>/<build_name>/daily.txt	registered ec2 AMI data for all daily builds ever
<suite>/<build_name>/daily.current.txt	registered ec2 AMI data for the most recent daily build
<suite>/<build_name>/released.txt	registered ec2 AMI data for all released builds
<suite>/<build_name>/released.current.txt	registered ec2 AMI data for the most recent released built

The downloadable image data files contain 7 tab delimited fields:

<suite>  <build_name> <label> <serial> <arch> <download_path> <suggested_name>
maverick server       daily   20100826 i386   server/maverick/20100826/maverick-server-uec-i386.tar.gz  ubuntu-maverick-daily-i386-server-20100826

Download path is relative to "http://uec-images.ubuntu.com/". Suggested name is the basename that this build would be published as in EC2.

The registered ec2 AMI data contain 10 tab delimited fields:

<suite>  <build_name> <label> <serial>    <root-store>   <arch> <region>  <ami>         <aki>        <ari>
maverick server       alpha3  20100803.2  instance-store i386   us-west-1 ami-ebdc8dae  aki-99a0f1dc <ari>

For many images, there may be no ari registered, so the field may be blank.

Ubuntu Documentation