Setting Up an MPICH2 Cluster with MPI-JAVA in Ubuntu (CONTINUED)

This is Part 2 of the documentation https://help.ubuntu.com/community/Mpich2ClusterWithMpijava

Creators: Amol Wagh & Cyril George (Interns at CDAC Pune from Jan 2011 to June 2011)

Email-Id: wagh.amol.m@gmail.com ,cyril.version1@gmail.com

12. Mounting /mirror in all other nodes

Now all we need to do is to mount the folder on the other nodes. This can be done manually each time like this:

mpiu@ub1:~$sudo mount ub0:/mirror /mirror
mpiu@ub2:~$sudo mount ub0:/mirror /mirror
mpiu@ub3:~$sudo mount ub0:/mirror /mirror

But it's better to change fstab in order to mount it on every boot. We do this by editing /etc/fstab and adding this line:

ub0:/mirror /mirror nfs
Notice the user name is same for all the machines but the hostname(node) is different.

13. Setting up SSH with no pass phrase for communication between nodes

After mounting all nodes, we need to create password-less login to all the nodes from the master nodes through SSH. To do this we do the following

mpiu@ub0:~$ ssh-keygen 

(You will then be asked the following questions. Simply hit “Enter” to answer them all)
Generating public/private rsa key pair.
Enter file in which to save the key (/mirror/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /mirror/.ssh/id_rsa.
Your public key has been saved in /mirror/.ssh/id_rsa.pub.
The key fingerprint is:
f6:61:a8:27:35:cf:4c:6d:13:22:70:cf:4c:c8:a0:23

mpiu@ub0:~$ ssh_copy_id -i ~/.ssh/id_rsa.pub remotehostname 

To summarize, a personal private/public key pair is generated using the ssh-keygen command. The public key is then copied onto a remote systems' .ssh/authorized_keys file. And you can now SSH to the remote systems's account without the use of a password.

Try "ssh hostname" and you should be able to login without a password prompt.

14. Running the java code using master node

Create a file with the names of the machines that you want to run your job on. This file may or may not include the local machine. Consider file name as hostFile and put following content-

hostname_of_machine_in_cluster:no_of_cores

e.g. 
 # This is a sample hostFile
   ub1:2     # The first 2 procs are scheduled to run here
   ub2:3     # The next 3 procs run on this host
   ub3:2     # The last 2 procs run on this host

By doing so,You are using the Hydra Process Manager to manage the processes across the nodes. For more information on Hydra Process Manager visit http://wiki.mcs.anl.gov/mpich2/index.php/Using_the_Hydra_Process_Manager
Compile your parallel java code using

mpiu@ub0:~$ javac fileName.java

To run your application on these nodes, use mpiexec:

mpiu@ub0:~$ mpiexec -n no_of_process -f hostFile java className

That's it !!!

References:

1. https://help.ubuntu.com/community/MpichCluster
2. http://wiki.mcs.anl.gov/mpich2/index.php/Using_the_Hydra_Process_Manager
3. http://blogs.translucentcode.org/mick/2003/09/18/shortest_passwordless_ssh_tutorial_ever
4. https://help.ubuntu.com/community/InstallingSoftware
5. http://www.ubuntugeek.com/nfs-server-and-client-configuration-in-ubuntu.html