An Open MPI Master & Servant Example

Building on the Getting started… post from last week I’ve knocked up a quick example showing one way to get your MPI processes to communicate with one another.
master_servant.c:

#include <stdio.h>
#include <mpi.h>
#include <unistd.h>
int main(int argc, char *argv[]) {
int numprocs, rank, namelen;
char processor_name[MPI_MAX_PROCESSOR_NAME];
MPI_Init(&argc, &argv);
MPI_Comm_size(MPI_COMM_WORLD, &numprocs);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Get_processor_name(processor_name, &namelen);
if ( rank == 0 ) {
printf( "[%02d/%02d %s]: I am the master\n", rank, numprocs, processor_name );
// Tell the servants to do something
} else {
printf( "[%02d/%02d %s]: I am a servant\n", rank, numprocs, processor_name );
// Wait for something to do
}
MPI_Finalize();
}

Build this with mpicc master_servant.c -o master_servant and run it, you should get something like:

[00/08 mpinode01]: I am the master
[01/08 mpinode01]: I am a servant
[02/08 mpinode01]: I am a servant
[03/08 mpinode01]: I am a servant
[04/08 mpinode02]: I am a servant
[05/08 mpinode02]: I am a servant
[06/08 mpinode02]: I am a servant
[07/08 mpinode02]: I am a servant

Ok, this means that based on the rank returned by MPI_Comm_rank we can decide which instance of the program is going to act as the “master” and which instance(s) are going to act as “servants” – pretty neat!
Next example, we build on this by getting the program instances to communicate with one another. Borrowed from the example found here.
master_servant2.c:

#include <stdio.h>
#include <string.h>
#include <mpi.h>
#include <unistd.h>
int main(int argc, char *argv[]) {
char idstr[32], buff[128];
int numprocs, rank, namelen, i;
char processor_name[MPI_MAX_PROCESSOR_NAME];
MPI_Status stat;
MPI_Init(&argc, &argv);
MPI_Comm_size(MPI_COMM_WORLD, &numprocs);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Get_processor_name(processor_name, &namelen);
// Based on example from https://wiki.inf.ed.ac.uk/pub/ANC/ComputationalResources/slides.pdf
if (rank == 0) {
// This is the rank-0 copy of the process
printf("We have %d processors\n", numprocs);
// Send each process a "Hello ... " string
for(i = 1; i < numprocs; i++) {
sprintf(buff, "Hello %d... ", i);
MPI_Send(buff, 128, MPI_CHAR, i, 0, MPI_COMM_WORLD);
}
// Go into a blocking-receive for each servant process
for(i = 1; i < numprocs; i++) {
MPI_Recv(buff, 128, MPI_CHAR, i, 0, MPI_COMM_WORLD, &stat);
printf("%s\n", buff);
}
} else {
// Go into a blocking-receive waiting
MPI_Recv(buff, 128, MPI_CHAR, 0, 0, MPI_COMM_WORLD, &stat);
// Append our identity onto the received string
sprintf(idstr, "Processor %d ", rank);
strcat(buff, idstr);
strcat(buff, "reporting!");
// Send the string back to the rank-0 process
MPI_Send(buff, 128, MPI_CHAR, 0, 0, MPI_COMM_WORLD);
}
MPI_Finalize();
}

Build this example with mpicc master_servant2.c -o master_servant2 and run it, you should get the following output:

We have 8 processors
Hello 1... Processor 1 reporting!
Hello 2... Processor 2 reporting!
Hello 3... Processor 3 reporting!
Hello 4... Processor 4 reporting!
Hello 5... Processor 5 reporting!
Hello 6... Processor 6 reporting!
Hello 7... Processor 7 reporting!
Hello 8... Processor 8 reporting!

Now you can use this master/servant technique to partition work across instances of your MPI-capable program.
Troubleshooting
If you get an error like this “[hostname][0,1,0][btl_tcp_endpoint.c:572:mca_btl_tcp_endpoint_complete_connect] connect() failed with errno=113” try shutting down iptables on the MPI nodes. It was a quick-fix for me, i am sure there is a ‘proper’ way to configure it though. Keep in mind its probably not a good idea to switch off iptables on a machine if its connected to the open internet, the machines I have used in this guide are all on an internal network.

3 thoughts on “An Open MPI Master & Servant Example”

  1. Nice examples!
    Your troubleshooting point is correct. The short version is that Open MPI works best if you have no firewall restrictions between the nodes that are used to launch your job (i.e., where mpirun was invoked) and the nodes where the MPI processes are running. Hence, shutting down iptables can be a reasonable solution — but your other security measures may also be important (e.g., you might not want to wholly shut down iptables if your machines are on the public internet and not otherwise protected; this is a local policy issue). Specifically: Open MPI uses random TCP ports between all of these nodes; don’t let them be restricted (e.g., by firewall rules) and you’ll be fine.

  2. These machines are all on my internal network, but its probably worth reiterating the security point in my post. Many thanks 🙂

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top