Some History

• 1980s - early 1990s: Parallel computing develops
  Along with it: dozens of competing, often incompatible and/or expensive tools for developing & running parallel software
• 1992: Needing a standard, MPI meetings begin

The nice thing about standards is that there are so many to choose from ~ Andrew Tanenbaum

MPI

The Message Passing Interface

MPI is not software; it is a specification Developers create what the users expect

Not an ISO or IEEE standard organically created by: vendors, researchers, developers, users

Goals: Practical, Portable, Efficient, Flexible

More History

• 1980s - early 1990s: Parallel computing develops
• 1992: Needing a standard, MPI meetings begin
• 1994: MPI 1.0 specification
• 1998: MPI 2.0 specification
• 2012: MPI 3.0 specification

MPI 3 contains 430 routines!

Hello, world

#include <mpi.h>
#include <stdio.h>

int main(int argc, char** argv) {
    MPI_Init(NULL, NULL); // Initialize MPI

    // get number of processes
    int world_size;
    MPI_Comm_size(MPI_COMM_WORLD, &world_size);

    // get my process's rank
    int world_rank;
    MPI_Comm_rank(MPI_COMM_WORLD, &world_rank);

    printf("Hello, world. - Love, process %d/%d \n",
           world_rank, world_size);

    MPI_Finalize();  // Clean-up
}

Compiling the code

OpenMPI provides wrappers for the C/Fortran compilers you are already using.

Commands: mpicc, mpifort

Call them as you would your usual complier

Running the code

Run manually: use mpirun

On UD's new supercomputer, the job scheduler (SLURM) is OpenMPI-aware...

srun or sbatch are all you need!

n copies of the program will run at the same time

(n: # processors you provision for your job)

Hello, Demo.

A Real Example

/*
 *  Let f(i) represent the first four digits
 *  after the decimal point of sin(i).
 *
 *  This program computes:
 *
 *          n
 * g(n) =  sum f(i)
 *         i=0
 *
 *   to four digits for a given n
 */

#include <math.h>
#include <stdlib.h>
#include <stdio.h>
#include <mpi.h>
#include <assert.h>

int nprocs; // number of processes
int myrank; // rank of this process

#define TAG 99 // TAG that will be used in all messages

/* Computes sum of f(i), where i takes on all the values
 * start, start+step, start+2*step, ...
 * that are less than stop.
 */
int sum(double start, double stop, double step) {
  int result = 0;

  for (double x = start; x < stop; x +=step) {
    double y = fabs(sin(x)); // 0.0<y<1.0
    int z = (int)(10000.0 * y); // 0<z<1000

    result = (result + z)%10000; // 0<result<1000
  }
  return result;
}
int main(int argc, char *argv[]) { 
  long stop;

  MPI_Init(&argc, &argv);
  MPI_Comm_rank(MPI_COMM_WORLD, &myrank);
  MPI_Comm_size(MPI_COMM_WORLD, &nprocs);

  if (myrank == 0) {
    assert(argc==2);
    stop = atol(argv[1]);
    assert(stop > 1);
  }

  MPI_Bcast(&stop, 1, MPI_LONG, 0, MPI_COMM_WORLD);

  int result = sum(myrank, (double)stop, (double)nprocs);

  if (myrank == 0) {
    int buf;

    for (int i=1; i<nprocs; i++) {
      MPI_Recv(&buf, 1, MPI_INT, i, TAG, MPI_COMM_WORLD, MPI_STATUS_IGNORE);
      result = (result + buf)%10000;
    }

    printf("Result: %d\n", result);
    fflush(stdout);
  }
  else {
    MPI_Send(&result, 1, MPI_INT, 0, TAG, MPI_COMM_WORLD);
  }

  MPI_Finalize();
  return 0;
}

Running it.