Setting Up an MPICH2 Cluster in Ubuntu

Creator: Omid Alemi

This guide describes how to building a simple MPICH cluster in ubuntu.

Before , you need an basic knowledge about mpich & clustering.

Here we have 4 nodes running ubuntu 7.04 with these host names: ub0,ub1,ub2,ub3;

1. Defining hostnames in etc/hosts/

Edit /etc/hosts like these:     localhost ub0 ub1 ub2 ub3

Note that the file shouldn't be like this:     localhost     ub0 ub0 ub1 ub2 ub3

or like this:     localhost     ub0 ub1 ub2 ub3

otherwise other hosts will try to connect to localhost when they try to reach ub0.

2. Installing NFS

NFS allows us to create a folder on the master node and have it synced on all the other nodes. This folder can be used to store programs. To Install NFS just run this in the master node's terminal:

omid@ub0:~$ sudo apt-get install nfs-kernel-server

3. Sharing Master Folder

Make a folder in all nodes, we'll store our data and programs in this folder.

omid@ub0:~$ sudo mkdir /mirror

And then we share the contents of this folder located on the master node to all the other nodes. In order to do this we first edit the /etc/exports file on the master node to contain the additional line

/mirror *(rw,sync)

This can be done using vim or by issuing this command:

omid@ub0:~$ sudo echo  /mirror *(rw,sync) >> /etc/exports

Note than we store out data and programs only in master node and other nodes will access them with NFS.

4. Mounting /master in nodes

Now all we need to do is to mount the folder on the other nodes. This can be done manually each time like this:

omid@ub1:~$sudo mount ub0:/mirror /mirror
omid@ub2:~$sudo mount ub0:/mirror /mirror
omid@ub3:~$sudo mount ub0:/mirror /mirror

But it's better to change fstab in order to mount it on every boot. We do this by editing /etc/fstab and adding this line:

ub0:/mirror /mirror nfs

5. Defining a user for running MPI programs

We define a user with same name and same userid in all nodes with a home directory in /mirror.

Here we name it "mpiu"! Also we change the owner of /mirror to mpiu:

omid@ub0:~$ sudo chown mpiu /mirror

6. Installing SSH Server

Run this command in all nodes in order to install OpenSSH Server

omid@ub0:~$ sudo apt­-get install openssh-server

7. Setting up SSH with no pass phrase for communication between nodes

First we login with our new user:

omid@ub0:~$ su - mpiu

Then we generate DSA key for mpiu:

mpiu@ub0:~$ ssh­-keygen ­-t dsa

Leave passphrase empty.

Next we add this key to authorized keys:

mpiu@ub0:~$ cd .ssh
mpiu@ub0:~/.ssh$ cat id_pub.dsa >> authorized_keys

As the home directory of mpiu in all nodes is the same (/mirror/mpiu) , there is no need to run these commands on all nodes.

To test SSH run:

mpiu@ub0:~$ ssh ub1 hostname

It should return remote hostname without asking for passphrase.

8. Installing GCC

Install build-essential package:

mpiu@ub0:~$ sudo apt-get install build-essential

9.Installing Other Compilers

Other prefered compilers should be installed before installing MPICH.

In this step we install other compilers such as Inter Fortran, SGI compiler , ... .

10. Installing MPICH2

You can install mpich2 using Synaptic by typing:

sudo apt-get install mpich2

Alternatively it can be installed from source...

Download MPICH2 source code from .

Extract .tar.bz2 file in /mirror. Also make a folder for MPICH installation.

mpiu@ub3:/mirror$ mkidr mpich2
mpiu@ub3:/mirror$ tar xvf mpich2-­1.0.5p3.tar.gz
mpiu@ub3:/mirror$ cd mpich2­-1.0.5p3
mpiu@ub3:/mirror/mpich2­-1.0.5p3$ ./configure --­prefix=/mirror/mpich2 
mpiu@ub3:/mirror/mpich2­-1.0.5p3$ make
mpiu@ub3:/mirror/mpich2­-1.0.5p3$ sudo make install

For more information about compilation see README file in source package.

After successfully compiling and installing mpich, add these lines to "/mirror/mpiu/.bashrc/"

export PATH=/mirror/mpich2/bin:$PATH
export PATH

Next we run this command in order to define MPICH installation path to SSH.

mpiu@ub0:~$ sudo echo /mirror/mpich2/bin >> /etc/environment

For testing our installation run:

mpiu@ub0:~$  which mpd
mpiu@ub0:~$  which mpiexec
mpiu@ub0:~$  which mpirun

11. setting up MPD

Create mpd.hosts in mpiu's home directory with nodes names:


and run :

mpiu@ub0:~$ echo secretword=something >> ~/.mpd.conf
mpiu@ub0:~$ chmod 600 ~/.mpd.conf

To test MPD run above commands. The output should be the current hostname.

mpiu@ub0:~$ mpd &
mpiu@ub0:~$ mpdtrace
mpiu@ub0:~$ mpdallexit

After all run mpd daemon:

mpiu@ub0:~$ mpdboot ­-n 4
mpiu@ub0:~$ mpdtrace

The output should be name of all nodes. If this doesn't succeed try running mpdcheck on all hosts to find possible errors in conf files (they will be marked with **).

There are some examples in "mpich2-1.0.5/examples", we'll run one :

mpiu@ub0:~$ mpiexec -n 4 cpi

That's it!

For more information visit:

MpiClusterUbuntu (last edited 2011-05-13 09:31:08 by Cyril George)