References

- Lectures notes from start of course through Oct 12. (Topical summary on course web page:
     http://www.cis.udel.edu/~saunders/courses/372/01f/
- Textbook: Chapters 1, 2, 3, 4, 5, 7(thru 7.4), 11,
- Individual Lab 1, Group Project 1.

Topic Coverage

- paradigms of parallel computing: data parallel, task parallel, pipelining
- parallel architectures: fine grain parallelism in a uniprocessor, SIMD, vector machines, array processors, MIMD, uniform shared memory, nonuniform shared memory, distributed memory, distributed shared memory
- SPMD versus MIMD style programming
- interconnection networks: topologies and advantages and disadvantages
- basic MPI program components and format and purpose of each component
- standard message passing in MPI: purpose of each field
- problems in parallel programming: deadlock, nondeterminism and races, load imbalance, communication overhead versus computation per process: what are these problems, symptoms, causes, approaches to dealing with them
- example application: numerical integration
- performance evaluation

Format of Exam

The exam is closed book, closed neighbor and you will have the full class period to work. You will be given a list of MPI commands with their parameters for reference. In general, the exam will be a combination of testing your basic knowledge and understanding of the concepts covered in class and application of the concepts. The questions will be of the form:

• Short answer.
  For example: Explain what "load imbalance" means.

• Possibly some multiple choice or items in list kinds of questions. For example: Circle the MPI commands in this list which are "collective communications". MPI_Comm_size, MPI_Recv, MPI_Reduce, MPI_Gather, MPI_Comm_group.

• Briefly describe a parallel algorithm for a particular problem, using a particular paradigm of parallel computing.
  For example: Describe a parallel message passing algorithm for integration using the trapezoid method.

• Explain what will happen when a particular code segment is executed.
  For example:

  // r has been initialized to my rank, p is the number of processes.
  int A = r;
  MPI_Send(&A, 1, MPI_INT, (r+1)%p, 0, MPI_COMM_WORLD);
  MPI_Recv(&A, 1, MPI_INT, (r-1+p)%p, 0, MPI_COMM_WORLD);
  if (r == 0) printf("%d\n", A);
  

  What happens in a buffered message MPI implementation (as on the Alphas)?
  What happens in a syncronized message MPI implementation?

• Write a short (5-10) line MPI program, given a list of MPI commands with parameters. For example: Each process has an array A, int A[N], of N numbers. So altogether there are N*p numbers in the arrays. Process 0 has a number K. Write a MPI program to return a single number, the total count of entries in the arrays A which are multiples of K. A[i] is a multiple of K if A[i]%K is 0.

Please answer each question as well as you can. Partial credit will be given when possible on any question in the exam.

How to Study

Review your lecture notes, labs, and textbook chapters. Rewrite some of the simple programs on your own, given a specification of what the program is supposed to do.