Overview

This tutorial covers UEC installation by adding the Eucalyptus packages to previously installed Ubuntu 10.04 servers where you want to have separate servers for the following services:

  • 1 Cloud Controller (CLC - eucalyptus-cloud) server
  • 1 Walrus Controller (WS3 - eucalyptus-walrus) server
  • N Cluster Controller (CC - eucalyptus-cc) + Storage Controller (EBS - eucalyptus-sc) servers
  • N Node Controller (NC - eucalyptus-nc) servers on a dedicated network connected to on of the CC+EBS server

Objective

From this tutorial, you will learn how to install, configure, register and perform several operations on a advanced UEC setup that results in a cloud as described in the above section. This topology is recommended if you would like to build a generic purpose scalable cloud. Our current testing indicate that a given cluster should limit itself to the management of up to 250 instances.

Tutorial

STEP 1: Prerequisites

To deploy a minimal cloud infrastructure, you’ll need at least 4 dedicated systems:

  • 1 CLC
  • 1 WS3
  • 1 CC + EBS
  • 1 NC

The following are recommendations, rather than fixed requirements. However, our experience in developing this documentation indicated the following suggestions.

Cloud Controller (CLC)

Hardware

Minimum

Suggested

Notes

CPU

1GHz

2 x 2GHz

For your cloud to scale it helps to have at least a dual core processor

Memory

512MB

2GB

the Java web front end benefits from lots of available memory

Disk

5400rpm IDE

5400rpm SATA

Disk Space

40GB

40GB

Networking

100Mbps

1000Mbps

Cloud controller may be subject to heavy traffic on a busy cloud setup

Walrus Controller (WS3)

Hardware

Minimum

Suggested

Notes

CPU

1GHz

2 x 2GHz

For your cloud to scale it helps to have at least a dual core processor

Memory

512MB

2GB

File serving benefits from the presence of memory for caching

Disk

5400rpm IDE

10000rpm SATA

File serving benefits from fast disk access

Disk Space

40GB

1TB

Disk space will condition image storage as well as user data

Networking

100Mbps

1000Mbps

Walrus may be subject to heavy traffic on a busy cloud setup and image needs to be copied over the network to NC at startup: faster network will mean faster initial instanciation time

Cluster + Storage Controllers (CC + EBS)

Hardware

Minimum

Suggested

Notes

CPU

1GHz

2 x 2GHz

For your cloud to scale it helps to have at least a dual core processor

Memory

512MB

2GB

Block storage serving benefits from the presence of memory for caching

Disk

5400rpm IDE

10000rpm SATA

Block storage serving benefits from fast disk access

Disk Space

40GB

1TB

Disk space will condition block stora capacity

2 Network interfaces

100Mbps

1000Mbps

Cluster will be subject to heavy traffic on a busy cloud setup as it will act as a router for all instances started on NC it Controls and will provide block storage (if needed) to them

Node(s) (NC)

The other system(s) are nodes, which will run the node controller (NC)

These systems will actually run the instances. You will need one or more systems with:

Hardware

Minimum

Suggested

Notes

CPU

VT extensions

VT, 64-bit, Multicore

64-bit can run both i386, and amd64 instances; by default, Eucalyptus will only run 1 VM per CPU core on a Node

Memory

1GB

4GB

additional memory means more, and larger guests

Disk

5400rpm IDE

7200rpm SATA or SCSI

Eucalyptus nodes are disk-intensive; I/O wait will likely be the performance bottleneck

Disk Space

40GB

100GB

images will be cached locally, Eucalyptus does not like to run out of disk space

Networking

100Mbps

1000Mbps

machine images are hundreds of MB, and need to be copied over the network to nodes

STEP 2a: Install the Cloud Controller Server

  1. Install Ubuntu 10.04 Server

  2. Update to the most current state in the Ubuntu archive:
    sudo apt-get update
    sudo apt-get dist-upgrade
  3. Update the time on your server using ntpdate
    sudo ntpdate ntp.ubuntu.com
  4. Install the following supporting packages on the server
    sudo apt-get install euca2ools openssh-server ntp
  1. Install the eucalyptus-cloud package on the server
    sudo apt-get install eucalyptus-cloud
    Answer debconf's questions as follow:
    • Configure postfix for internet delivery

  2. Make sure process eucalyptus-cloud is running
    ps -ef | grep eucalyptus-cloud

STEP 2b: Install the Cluster Controller Server(s)

  1. Install Ubuntu 10.04 Server

  2. Update to the most current state in the Ubuntu archive:
    sudo apt-get update
    sudo apt-get dist-upgrade
  3. Update the time on your server using ntpdate
    sudo ntpdate ntp.ubuntu.com
  4. Install the following supporting packages on the server
    sudo apt-get install euca2ools openssh-server ntp
  1. Install the eucalyptus-cc and eucalyptus-sc packages on the server
    sudo apt-get install eucalyptus-cc eucalyptus-sc
    Answer debconf's questions as follow:
    • Configure postfix for internet delivery

    • Name your cluster
      • e.g. cluster1
    • Add a list of available IP addresses on your external (public) network
      • e.g. 192.168.1.200-192.168.1.249
  2. Configure the second network interface
    sudo apt-get install bridge-utils vtun
    interface=eth1
    bridge=br1
    sudo sed -i "s/^iface $interface inet \(.*\)$/iface $interface inet manual\n\nauto $bridge\niface $bridge inet \1/" /etc/network/interfaces
    sudo tee -a /etc/network/interfaces <<EOF
            bridge_ports $interface
            bridge_fd 9
            bridge_hello 2
            bridge_maxage 12
            bridge_stp off
    EOF
    sudo /etc/init.d/networking restart
  3. Edit /etc/eucalyptus/eucalyptus.conf so that

    • br1 is the internal (private) interface and the bridge interface
    • eth0 the external (public) interface
    • you provide the correct address of the cloud controller (CLC) that you just installed in place of <CLC_ipaddress>

    by editing the following 3 variables, the third one (VNET_CLOUDIP) needing to be added manually:
    VNET_PUBINTERFACE="eth0"
    VNET_PRIVINTERFACE="br1"
    VNET_CLOUDIP="<CLC_ipaddress>"
  4. Edit /etc/eucalyptus/eucalyptus-ipaddr.conf so that the following variable are set to the internal (private) ip address of the server

    CC_IP_ADDR="<CC_ipaddress>"
    SC_IP_ADDR="<CC_ipaddress>"
  5. Restart the service with a config reinitialisation
    sudo restart eucalyptus-cc CLEAN=1
    sudo restart eucalyptus-sc
  6. Make sure processes eucalyptus-cc and eucalyptus-sc are running
    ps -ef | grep eucalyptus-

STEP 2c: Install the Walrus Controller Server

  1. Install Ubuntu 10.04 Server

  2. Update to the most current state in the Ubuntu archive:
    sudo apt-get update
    sudo apt-get dist-upgrade
  3. Update the time on your server using ntpdate
    sudo ntpdate ntp.ubuntu.com
  4. Install the following supporting packages on the server
    sudo apt-get install euca2ools openssh-server ntp
  1. Install the eucalyptus-walrus package on the server
    sudo apt-get install eucalyptus-walrus

STEP 3: Install and Configure the Node Controller(s)

Installing the Eucalyptus Node Service on a running Ubuntu system consists of doing the following on one or more systems:

  1. Install Ubuntu 10.04 Server

  2. Update to the most current state in the Ubuntu archive:
    sudo apt-get update
    sudo apt-get dist-upgrade
  3. Update the time on your server using ntpdate
    sudo ntpdate ntp.ubuntu.com
  4. Install the following supporting packages on the server
    sudo apt-get install euca2ools openssh-server ntp
  1. Install the eucalyptus-nc package
    sudo apt-get install eucalyptus-nc
  2. Configure the system's primary ethernet interface as a bridge. The node controller will attach virtual network interfaces to this bridge for VM that is started before it to enable network connectivity.
    • Note: Remember the name of your node's bridge device (we assume the name of your bridge device is "br0" for the rest of this document).
    • For details on configuring a bridge, see: http://doc.ubuntu.com/ubuntu/serverguide/C/network-configuration.html

    • The following script should configure your bridge correctly in most setups:
      interface=eth0
      bridge=br0
      sudo sed -i "s/^iface $interface inet \(.*\)$/iface $interface inet manual\n\nauto br0\niface $bridge inet \1/" /etc/network/interfaces
      sudo tee -a /etc/network/interfaces <<EOF
              bridge_ports $interface
              bridge_fd 9
              bridge_hello 2
              bridge_maxage 12
              bridge_stp off
      EOF
      sudo /etc/init.d/networking restart
  3. Configure /etc/eucalyptus/eucalyptus.conf with the name of the bridge, and restart the node controller:

    sudo sed -i "s/^VNET_BRIDGE=.*$/VNET_BRIDGE=$bridge/" /etc/eucalyptus/eucalyptus.conf
    sudo /etc/init.d/eucalyptus-nc restart
    • Note that there are several ways to configure a node to have a bridge as its primary interface, depending on the configuration of your machine. We show an example set of steps here but you will need to take care to ensure that this example configuration does not conflict with your local configuration if you wish to use it.

  4. Finally, you need to install the Cluster Controller's eucalyptus user's public ssh key into the Node Controller's eucalyptus user's authorized_keys file. The easiest way to do this:

    • On the Node Controller, temporarily set a password for the eucalyptus user:

      sudo passwd eucalyptus
    • Then, on the Cluster Controller:
      sudo -u eucalyptus ssh-copy-id -i ~eucalyptus/.ssh/id_rsa.pub eucalyptus@<IP_OF_NODE>
    • You can now remove the password of the eucalyptus account on the Node:

      sudo passwd -d eucalyptus

STEP 4: Register the Cluster, Storage, and Walrus Servers

The cloud controller must be aware the storage (SC, WS3) and of the clusters controllers, so you will need to allow password-less authentication between the CLC and the other server and then register them.

SSH Key Authentication Setup

An essential step is to enable password less authentication for the eucalyptus user on all the controllers.

Repeat the following operation for each walrus (WS3) and cluster (CC) controller servers:

  1. On the target (CC or WS3) Controller, temporarily set a password for the eucalyptus user:

    sudo passwd eucalyptus
  2. Then, on the Cloud Controller (CLC):
    sudo -u eucalyptus ssh-copy-id -i ~eucalyptus/.ssh/id_rsa.pub eucalyptus@<IP_OF_TARGET>
  3. You can now remove the password of the eucalyptus account on the target:

    sudo passwd -d eucalyptus

Registration

To register the controllers on the CLC:

  1. Register the WS3 server:
    sudo euca_conf --no-rsync --register-walrus <WS3_ipaddresss>
  2. Then for each of your CC:
    sudo euca_conf --no-rsync --register-cluster <CC_name> <CC_ipaddress>
    sudo euca_conf --no-rsync --register-sc <CC_name> <CC_ipaddress>

    Note: If you get prompted for passwords, or receive errors from scp, you may need to revisit the key synchronization instructions at SSH Key Authentication Setup

  3. Then edit the /etc/eucalyptus/eucalyptus-ipaddr.conf to fill the 3 following variables appropriately with the ip address of the respective service, replacing the "$addr" value. If you have multiple CC and SC, you can specify multiple IP addresses separating them with a space.

    CC_IP_ADDR="$addr"
    WALRUS_IP_ADDR="$addr"
    SC_IP_ADDR="$addr"
  4. Finally restart the eucalyptus service on the CLC
     sudo service eucalyptus restart

Verify Registration

Verify that the registration process was successful by running the following commands

  • sudo euca_conf --list-walruses
    sudo euca_conf --list-scs
    sudo euca_conf --list-clusters

STEP 5: Register the Node(s)

As of Ubuntu 10.04 LTS, all component registration should be automatic, assuming:

  1. Public SSH keys have been exchanged properly
  2. The services are configured properly
  3. The services are publishing their existence
  4. The appropriate uec-component-listener is running
  5. Verify Registration.

Steps a to e should only be required if you're using the UEC/PackageInstall method. Otherwise, if you are following the UEC/CDInstall method, these steps should already be completed automatically for you, and therefore you can skip a to e.

a.

Exchange Public SSH Keys

The Cloud Controller's eucalyptus user needs to have SSH access to the Walrus Controller, Cluster Controller, and Storage Controller as the eucalyptus user.

Install the Cloud Controller's eucalyptus user's public ssh key by:

  • On the target controller, temporarily set a password for the eucalyptus user:

    sudo passwd eucalyptus
  • Then, on the Cloud Controller:
    sudo -u eucalyptus ssh-copy-id -i /var/lib/eucalyptus/.ssh/id_rsa.pub eucalyptus@<IP_OF_NODE>
  • You can now remove the password of the eucalyptus account on the target controller, if you wish:

  • sudo passwd -d eucalyptus

b.

Configure the Services

On the Cloud Controller:

  • For the Cluster Controller Registration:

    • Define the shell variable CC_NAME in /etc/eucalyptus/eucalyptus-cc.conf

    • Define the shell variable CC_IP_ADDR in /etc/eucalyptus/eucalyptus-ipaddr.conf, as a space separated list of one or more IP addresses.

  • For the Walrus Controller Registration:

    • Define the shell variable WALRUS_IP_ADDR in /etc/eucalyptus/eucalyptus-ipaddr.conf, as a single IP address.

On the Cluster Controller:

  • For Storage Controller Registration:

    • Define the cluster name in the shell variable CC_NAME in /etc/eucalyptus/eucalyptus-cc.conf

    • Define the shell variable SC_IP_ADDR in /etc/eucalyptus/eucalyptus-ipaddr.conf, as a space separated list of one or more IP addresses.

c.

Publish

Now start the publication services.

  • Walrus Controller:

    sudo start eucalyptus-walrus-publication
  • Cluster Controller:

    sudo start eucalyptus-cc-publication
  • Storage Controller:

    sudo start eucalyptus-sc-publication
  • Node Controller

    sudo start eucalyptus-nc-publication

d.

Start the Listener

On the Cloud Controller and the Cluster Controller(s), run:

sudo start uec-component-listener

e.

Verify Registration

cat /var/log/eucalyptus/registration.log
2010-04-08 15:46:36-05:00 | 24243 -> Calling node cluster1 node 10.1.1.75
2010-04-08 15:46:36-05:00 | 24243 -> euca_conf --register-nodes returned 0
2010-04-08 15:48:47-05:00 | 25858 -> Calling walrus Walrus 10.1.1.71
2010-04-08 15:48:51-05:00 | 25858 -> euca_conf --register-walrus returned 0
2010-04-08 15:49:04-05:00 | 26237 -> Calling cluster cluster1 10.1.1.71
2010-04-08 15:49:08-05:00 | 26237 -> euca_conf --register-cluster returned 0
2010-04-08 15:49:17-05:00 | 26644 -> Calling storage cluster1 storage 10.1.1.71
2010-04-08 15:49:18-05:00 | 26644 -> euca_conf --register-sc returned 0

STEP 6: Obtain Credentials

After installing and booting the Cloud Controller, users of the cloud will need to retrieve their credentials. This can be done either through a web browser, or at the command line.

From a Web Browser

  1. From your web browser (either remotely or on your Ubuntu server) access the following URL:
    https://<cloud-controller-ip-address>:8443/

    Important! You must use a secure connection, so make sure you use "https" not "http" in your URL. You will get a security certificate warning. You will have to add an exception to view the page. If you do not accept it you will not be able to view the Eucalyptus configuration page.

  2. Use username 'admin' and password 'admin' for the first time login (you will be prompted to change your password).
  3. Then follow the on-screen instructions to update the admin password and email address.
  4. Once the first time configuration process is completed, click the 'credentials' tab located in the top-left portion of the screen.

  5. Click the 'Download Credentials' button to get your certificates
  6. Save them to ~/.euca

  7. Unzip the downloaded zipfile into a safe location (~/.euca)
    unzip -d ~/.euca mycreds.zip

From a Command Line

  1. Alternatively, if you are on the command line of the Cloud Controller, you can run:
    mkdir -p ~/.euca
    chmod 700 ~/.euca
    cd ~/.euca
    sudo euca_conf --get-credentials mycreds.zip
    unzip mycreds.zip
    ln -s ~/.euca/eucarc ~/.eucarc
    cd -

Extracting and Using Your Credentials

Now you will need to setup EC2 API and AMI tools on your server using X.509 certificates.

  1. Install the required cloud user tools:
    sudo apt-get install euca2ools
  2. To validate that everything is working correctly, get the local cluster availability details:
    . ~/.euca/eucarc
    euca-describe-availability-zones verbose
    AVAILABILITYZONE   myowncloud                 192.168.1.1
    AVAILABILITYZONE   |- vm types                free / max   cpu   ram  disk
    AVAILABILITYZONE   |- m1.small                0004 / 0004   1    192     2
    AVAILABILITYZONE   |- c1.medium               0004 / 0004   1    256     5
    AVAILABILITYZONE   |- m1.large                0002 / 0002   2    512    10
    AVAILABILITYZONE   |- m1.xlarge               0002 / 0002   2   1024    20
    AVAILABILITYZONE   |- c1.xlarge               0001 / 0001   4   2048    20

STEP 7: Install an image from the store

The following is by far the simplest way to install an image. However, advanced users may be interested in learning how to Bundle their own image.

The simplest way to add an image to UEC is to install it from the Image Store on the UEC web interface.

  1. Access the web interface at the following URL (Make sure you specify https):
    https://<cloud-controller-ip-address>:8443/
  2. Enter your login and password (if requested, as you may still be logged in from earlier)
  3. Click on the Store tab

    Private-store-cr.png

  4. Browse available images
  5. Click on install for the image you want

Once the image has been downloaded and installed, you can click on "How to run?" that will be displayed below the image button to view the command to execute to instantiate (start) this image. The image will also appear on the list given on the Image tab.

  • private-images-cr.png

STEP 8: Run an Image

There are multiple ways to instantiate an image in UEC:

  • Use the command line
  • Use one of the UEC compatible management tools such as Landscape
  • Use the ElasticFox extension to Firefox

Here we will describe the process from the command line:

  1. Before running an instance of your image, you should first create a keypair (ssh key) that you can use to log into your instance as root, once it boots. The key is stored, so you will only have to do this once. Run the following command:
    if [ ! -e ~/.euca/mykey.priv ]; then
        mkdir -p -m 700 ~/.euca
        touch ~/.euca/mykey.priv
        chmod 0600 ~/.euca/mykey.priv
        euca-add-keypair mykey > ~/.euca/mykey.priv
    fi

    Note: You can call your key whatever you like (in this example, the key is called 'mykey'), but remember what it is called. If you forget, you can always run euca-describe-keypairs to get a list of created keys stored in the system.

  2. You must make sure to source ~/.euca/eucarc before you run any of the eucatools. It is probably best to add this to the bottom of your .bashrc script.
  3. You must also allow access to port 22 in your instances:
    euca-authorize default -P tcp -p 22 -s 0.0.0.0/0
  4. Next, you can create instances of your registered image:
    euca-run-instances $EMI -k mykey -t m1.small

    Note: If you receive an error regarding image_id, you may find it by viewing Images page or click "How to Run" on the Store page to see the sample command.

  5. The first time you run an instance, the system will be setting up caches for the image from which it will be created. This can often take some time the first time an instance is run given that VM images are usually quite large. To monitor the state of your instance, run:
    watch -n5 euca-describe-instances
    In the output, you should see information about the instance, including its state. While first-time caching is being performed, the instance's state will be 'pending'.
  6. When the instance is fully started, the above state will become 'running'. Look at the IP address assigned to your instance in the output, then connect to it:
    IPADDR=$(euca-describe-instances | grep $EMI | grep running | tail -n1 | awk '{print $4}')
    ssh -i ~/.euca/mykey.priv ubuntu@$IPADDR
  7. And when you are done with this instance, exit your SSH connection, then terminate your instance:
    INSTANCEID=$(euca-describe-instances | grep $EMI | grep running | tail -n1 | awk '{print $2}')
    euca-terminate-instances $INSTANCEID

More Information

How to use the Storage Controller

Controlling eucalyptus services:

  • sudo service eucalyptus [start|stop|restart] (on the CLC/CC/SC/Walrus side)
  • sudo service eucalyptus-nc [start|stop|restart] (on the Node side)

Locations of some important files:

  • Log files:

    • /var/log/eucalyptus

  • Configuration files:

    • /etc/eucalyptus

  • Database:

    • /var/lib/eucalyptus/db

  • Keys

    • /var/lib/eucalyptus

    • /var/lib/eucalyptus/.ssh

Notes:

  • Don't forget to source your ~/.euca/eucarc before running the client tools.

Links:

Glossary

The Ubuntu Enterprise Cloud documentation uses terminology that might be unfamiliar to some readers. This page is intended to provide a glossary of such terms and acronyms.

  • Cloud - A federated set of physical machines that offer computing resources through virtual machines, provisioned and recollected dynamically.

  • Cloud Controller (CLC) - Eucalyptus component that provides the web UI (an https server on port 8443), and implements the Amazon EC2 API. There should be only one Cloud Controller in an installation of UEC. This service is provided by the Ubuntu eucalyptus-cloud package.

  • Cluster - A collection of nodes, associated with a Cluster Controller. There can be more than one Cluster in an installation of UEC. Clusters are sometimes physically separate sets of nodes. (e.g. floor1, floor2, floor2).

  • Cluster Controller (CC) - Eucalyptus component that manages collections of node resources. This service is provided by the Ubuntu eucalyptus-cc package.

  • EBS - Elastic Block Storage. http://aws.amazon.com/ebs/

  • EC2 - Elastic Compute Cloud. Amazon's pay-by-the-hour, pay-by-the-gigabyte public cloud computing offering.

  • EKI - Eucalyptus Kernel Image.

  • EMI - Eucalyptus Machine Image.

  • ERI - Eucalyptus Ramdisk Image.

  • Eucalyptus - Elastic Utility Computing Architecture for Linking Your Programs To Useful Systems. An open source project originally from the University of California at Santa Barbara, now supported by Eucalyptus Systems, a Canonical Partner.

  • Front-end - Physical machine hosting one (or more) of the high level Eucalyptus components (cloud, walrus, storage controller, cluster controller).

  • Node - A node is a physical machine that's capable of running virtual machines, running a node controller. Within Ubuntu, this generally means that the CPU has VT extensions, and can run the KVM hypervisor.

  • Node Controller (NC) - Eucalyptus component that runs on nodes which host the virtual machines that comprise the cloud. This service is provided by the Ubuntu package eucalyptus-nc.

  • S3 - Simple Storage Service. Amazon's pay-by-the-gigabyte persistent storage solution for EC2. http://aws.amazon.com/s3/

  • Storage Controller (SC) - Eucalyptus component that manages dynamic block storage services (EBS). Each 'cluster' in a Eucalyptus installation can have its own Storage Controller. This component is provided by the 'eucalyptus-sc' package.

  • UEC - Ubuntu Enterprise Cloud. Ubuntu's cloud computing solution, based on Eucalyptus.

  • VM - Virtual Machine.

  • VT - Virtualization Technology. An optional feature of some modern CPUs, allowing for accelerated virtual machine hosting.

  • Walrus - Eucalyptus component that implements the Amazon S3 API, used for storing VM images and user storage using S3 bucket put/get abstractions.

UEC/PackageInstallSeparate (last edited 2010-06-30 17:55:58 by kirkland)