Language

Introduction

There is now a kernel module (aoe), tools (aoetools) and server software (vblade) which allows anyone to set up an ATA over Ethernet storage network... it is very easy to do in Ubuntu.

This software was available in Dapper 6.06 LTS, but has also been tested in Feisty (beta).

WARNING: These commands need to be run with root privileges (sudo -i). Do this at your own risk.

WARNING: Don't mount the target from more than one client or filesystem corruption will occur. To do this something like GFS is needed.

Required Packages

With the Universe repository enabled..

On the Ubuntu host that you wish to be the disk server (due to a bug in Ubuntu 8.04 LTS, you will need vblade 14-1ubuntu1.1 for this to work. See 223440 for details):

  (server)# apt-get install vblade

On the Ubuntu host that you wish to mount the disk on:

  (client)# apt-get install aoetools

To set the interface up

It's important to note that no IP address is needed to get this working.

 # ip link set eth0 up

To create an AoE device

To test that everything is working correctly, we created and used a 1 Mbyte file (vblade0). Any other block device or file could be used.

On the server:

  (server)# dd if=/dev/zero of=vblade0 count=1 bs=1M
  (server)# vblade 1 1 eth0 vblade0
  ioctl returned 0
  1048576 bytes
  pid 25000: e1.1, 2048 sectors

The deamon ('vbladed') could be used instead of 'vblade', which detaches the process from the console and send log messages to the system logs.

Using AoE

The 'aoe' module should already be automatically loaded when the 'aoetools' package is installed. Please note that you will not be able to discover AoE devices on the localhost unless you specify lo to vblade.

If this hasn't occured, insert the module into the linux kernel.

  (client)# modprobe aoe

optionally the list of interfaces can be specified while loading the module

   (client)# modprobe aoe aoe_iflist="eth0 eth1"

This can automatically be performed at startup, by entering a line into the /etc/modules file.

In Ubuntu 6.06 (or any systems without udev), create the 'etherd' directory:

  (client)# mkdir /dev/etherd

In Ubuntu 8.04, add the necessary rules to udev (this is not needed in 9.04 and later, see 234033):

   (client)# cat > /etc/udev/rules.d/60-aoe.rules << EOM
SUBSYSTEM=="aoe", NAME="etherd/%k", GROUP="disk", MODE="0220"
SUBSYSTEM=="aoe", KERNEL=="err", MODE="0440"
EOM
   (client)# /etc/init.d/udev restart

Next, access the devices:

  (client)# aoe-interfaces eth0
  (client)# aoe-discover
  (client)# aoe-stat
      e1.1         0.001GB   eth0 up 

The block device is then located in /dev/etherd/e1.1 and can be treated like any other block device.

Create a filesystem:

  (client)# mke2fs /dev/etherd/e1.1

Create a mount point and mount the file system:

  (client)# mkdir /mnt/e1.1
  (client)# mount /dev/etherd/e1.1 /mnt/e1.1

This can be made to automatically mount on reboot, by entering a configuration line in /etc/fstab (untested).

If the AoE device disappears or changes size use the aoe-revalidate command:

   (client)# aoe-revalidate e1.1

Alternatively, you can re-load the aoe module completely.

   (client)# rmmod aoe
   (client)# modprobe aoe

Since aoe6-15, aoe drivers have supported the aoe-revalidate command. See the aoe-revalidate manpage for more information.

References

• http://support.coraid.com/support/linux/EtherDrive-2.6-HOWTO.html

• http://en.wikipedia.org/wiki/ATA_over_Ethernet

• http://en.wikipedia.org/wiki/Global_File_System

CategoryHardware CategoryNetworking