Setting Up an MPICH2 Cluster in Ubuntu
Creator: Omid Alemi
This guide describes how to building a simple MPICH cluster in ubuntu.
Before , you need an basic knowledge about mpich & clustering.
Here we have 4 nodes running ubuntu 7.04 with these host names: ub0,ub1,ub2,ub3;
1. Defining hostnames in etc/hosts/
Edit /etc/hosts like these:
127.0.0.1 localhost 192.168.133.100 ub0 192.168.133.101 ub1 192.168.133.102 ub2 192.168.133.103 ub3
Note that the file shouldn't be like this:
127.0.0.1 localhost 127.0.1.1 ub0 192.168.133.100 ub0 192.168.133.101 ub1 192.168.133.102 ub2 192.168.133.103 ub3
or like this:
127.0.0.1 localhost 127.0.1.1 ub0 192.168.133.101 ub1 192.168.133.102 ub2 192.168.133.103 ub3
otherwise other hosts will try to connect to localhost when they try to reach ub0.
2. Installing NFS
NFS allows us to create a folder on the master node and have it synced on all the other nodes. This folder can be used to store programs. To Install NFS just run this in the master node's terminal:
omid@ub0:~$ sudo apt-get install nfs-kernel-server
3. Sharing Master Folder
Make a folder in all nodes, we'll store our data and programs in this folder.
omid@ub0:~$ sudo mkdir /mirror
And then we share the contents of this folder located on the master node to all the other nodes. In order to do this we first edit the /etc/exports file on the master node to contain the additional line
/mirror *(rw,sync)
This can be done using vim or by issuing this command:
omid@ub0:~$ sudo echo /mirror *(rw,sync) >> /etc/exports
Note than we store out data and programs only in master node and other nodes will access them with NFS.
4. Mounting /master in nodes
Now all we need to do is to mount the folder on the other nodes. This can be done manually each time like this:
omid@ub1:~$sudo mount ub0:/mirror /mirror omid@ub2:~$sudo mount ub0:/mirror /mirror omid@ub3:~$sudo mount ub0:/mirror /mirror
But it's better to change fstab in order to mount it on every boot. We do this by editing /etc/fstab and adding this line:
ub0:/mirror /mirror nfs
5. Defining a user for running MPI programs
We define a user with same name and same userid in all nodes with a home directory in /mirror.
Here we name it "mpiu"! Also we change the owner of /mirror to mpiu:
omid@ub0:~$ sudo chown mpiu /mirror
6. Installing SSH Server
Run this command in all nodes in order to install OpenSSH Server
omid@ub0:~$ sudo apt-get install openssh-server
7. Setting up SSH with no pass phrase for communication between nodes
First we login with our new user:
omid@ub0:~$ su - mpiu
Then we generate DSA key for mpiu:
mpiu@ub0:~$ ssh-keygen -t dsa
Leave passphrase empty.
Next we add this key to authorized keys:
mpiu@ub0:~$ cd .ssh mpiu@ub0:~/.ssh$ cat id_pub.dsa >> authorized_keys
As the home directory of mpiu in all nodes is the same (/mirror/mpiu) , there is no need to run these commands on all nodes.
To test SSH run:
mpiu@ub0:~$ ssh ub1 hostname
It should return remote hostname without asking for passphrase.
8. Installing GCC
Install build-essential package:
mpiu@ub0:~$ sudo apt-get install build-essential
9.Installing Other Compilers
Other prefered compilers should be installed before installing MPICH.
In this step we install other compilers such as Inter Fortran, SGI compiler , ... .
10. Installing MPICH2
You can install mpich2 using Synaptic by typing:
sudo apt-get install mpich2
Alternatively it can be installed from source...
Download MPICH2 source code from http://www-unix.mcs.anl.gov/mpi/mpich .
Extract .tar.bz2 file in /mirror. Also make a folder for MPICH installation.
mpiu@ub3:/mirror$ mkidr mpich2 mpiu@ub3:/mirror$ tar xvf mpich2-1.0.5p3.tar.gz mpiu@ub3:/mirror$ cd mpich2-1.0.5p3 mpiu@ub3:/mirror/mpich2-1.0.5p3$ ./configure --prefix=/mirror/mpich2 mpiu@ub3:/mirror/mpich2-1.0.5p3$ make mpiu@ub3:/mirror/mpich2-1.0.5p3$ sudo make install
For more information about compilation see README file in source package.
After successfully compiling and installing mpich, add these lines to "/mirror/mpiu/.bashrc/"
export PATH=/mirror/mpich2/bin:$PATH export PATH LD_LIBRARY_PATH="/mirror/mpich2/lib:$LD_LIBRARY_PATH" export LD_LIBRARY_PATH
Next we run this command in order to define MPICH installation path to SSH.
mpiu@ub0:~$ sudo echo /mirror/mpich2/bin >> /etc/environment
For testing our installation run:
mpiu@ub0:~$ which mpd mpiu@ub0:~$ which mpiexec mpiu@ub0:~$ which mpirun
11. setting up MPD
Create mpd.hosts in mpiu's home directory with nodes names:
ub3 ub2 ub1 ub0
and run :
mpiu@ub0:~$ echo secretword=something >> ~/.mpd.conf mpiu@ub0:~$ chmod 600 ~/.mpd.conf
To test MPD run above commands. The output should be the current hostname.
mpiu@ub0:~$ mpd & mpiu@ub0:~$ mpdtrace mpiu@ub0:~$ mpdallexit
After all run mpd daemon:
mpiu@ub0:~$ mpdboot -n 4 mpiu@ub0:~$ mpdtrace
The output should be name of all nodes. If this doesn't succeed try running mpdcheck on all hosts to find possible errors in conf files (they will be marked with **).
There are some examples in "mpich2-1.0.5/examples", we'll run one :
mpiu@ub0:~$ mpiexec -n 4 cpi
That's it!
For more information visit: