Course Objectives:

• DESCRIBE the major functions and approaches to parallelism used in a large parallel program, through program inspection and research on an application domain,
  
• APPLY profiling tools to identify the regions of a large time-consuming program that are most conducive to increased performance through parallelization,
  
• SELECT and apply appropriate parallelization constructs to a large program to create a correct parallel version to exploit a state-of-the-art parallel computing environment,
  
• ASSESS the correctness of a parallel program,
  
• APPLY available debugging methods to detect and correct errors in an erroneous parallel program,
  
• ASSESS the performance of parallel programs run with different parameters of input sizes and numbers of processors,
  
• VERBALLY AND ORALLY CRITIQUE a parallelization effort and its resulting performance results to non-experts,
  
• COORDINATE and cooperatively participate in a team similar to industrial and government research labs working on a realistic program parallelization problem

Course Topics:

• Introduction - What is parallel computing?
• Overview of parallel architectures and parallel programming models
• How do I write a simple parallel program?
• The MIMD and SPMD models
• What are the problems unique to parallel programs?
• How do I deal with these problems?
• How do I debug parallel programs?
• How do I measure the goodness of my parallel program?
• The process of designing good parallel programs:
  • How do I break up the problem?
  • Some basic techniques of parallel algorithm design
  • How do I get processes to pass data around in effective ways?
  • How much overhead have I created for the sake of parallelism?
  • How can I improve the parallel program in terms of communication overhead?
• How do I evaluate the performance of my parallel algorithm/program?

MPI Concepts:

• Basic Send and Recv
• Collective Communication
• Communicators and Groups
• Topologies
• Derived Data Types
• Advanced Point-to-Point Communication
• Timing and Profiling MPI Programs

Instructor: Lori Pollock pollock@cis.udel.edu 101D Smith Hall (831-1953)
Office Hours: TTh 11:00-12:00 PM, and by appointment.

Ananth Grama, Anshul Gupta, George Karypis, Vipin Kumar, Introduction to Parallel Computing. Second Edition. Addison-Wesley, 2003 (ISBN 0 201 64865 2).

References:

• Michael J. Quinn, Parallel Programming in C with MPI and OpenMP, McGraw Hill, 2004.
• Kernighan and Ritchie, The C Programming Language, Prentice Hall, 1988.
• Stroustrop, The C++ Programming Languages, Addison Wesley, 1997.
• Deitel and Deitel, C How to Program, Prentice Hall, 1994.
• Snir, Otto, Huss-Lederman, Walker, and Dongarra, MPI The Complete Reference, The MIT Press, 1994.
• William Gropp, Ewing Lusk, and Anthony Skjellum, Using MPI, The MIT Press, 1994.

Programming Environment and Computer Usage:

We will write parallel programs primarily using MPI (Message Passing Interface). MPI is a library of routines that allows programs running in parallel to talk to each other, and send data back and forth. These programs will be compiled and executed on a dedicated network of linux PCs on the CIS network. You must obtain an account on this cluster as soon as possible. Go to htto://www.eecis.udel.edu and click on "Apply for Account". Ask for an academic account. When using the cluster login to "porsche.cis.udel.edu".

• (52%) Laboratory exercises and group projects.
  • (28%) Four Individual programming exercises.
  • (24%) Two team projects (teams of two-three).
• (8%) Weekly Quizzes
• (10%) Short team oral presentation on a real-life application of parallelism. You will search the web for a demonstration of an application that depends on parallelism to achieve its goal, and give a short (15 minute) presentation describing the application and how it uses parallelism, how parallelism is achieved, and why parallelism is so crucial to its success.
• (15%) Midterm Examination.
• (15%) Final Examination.

Email Questions:

Labs must be given to the instructor by the start of class on the due date, in order to have no points deducted for lateness. The due dates are to be taken seriously and you should not expect them to be extended. The pace of work is implicit in the due dates and necessary if you expect to finish by the end of the semester. Homeworks to be graded should be turned in at the start of class on the specified due date. NO late programs or homeworks will be accepted FOR FULL CREDIT without discussion with me prior to the due date. If you can not reach me, leave a message on my voicemail. All other assignments not delivered by the due date are considered late.

My philosophy on late assignments is: (1) Everyone should try their best to complete all assignments by the specified due date. (2) People who work conscientiously to make the deadlines should be rewarded for their promptness and sacrifice of sleep. Thus, allowing others to hand in late assignments without some penalty is not fair to these people. However, there are various circumstances that may prevent you from completing an assignment by the due date. Allowing no late assignments would not give you much incentive to continue to work on the assignment, which is a major source of learning in this course. Thus, I believe late assignments are better than no assignment.

If you are dissatisfied with a grade on a homework, programming assignment, or exam, you should consult the instructor directly within a week of the day the graded assignment was returned to you. No regrade requests will be considered after this week period.

Posting Grades:

With your permission, grades will be posted periodically (by your secret code) on the course web page. Questions about accuracy of recorded grades should be addressed to me.

Policy on Academic Dishonesty:

You are permitted to consult with other students and professors on any conceptual problems or for debugging assistance on all programming assignments. Any evidence of collaboration other than this kind will be handled as stated in the Official Student Handbook of the University of Delaware. If you are in doubt regarding the requirements, please consult with me before you complete any requirement of this course.