CPE542 - Parallel Processing (Fall 2000)

Professor: Emil Jovanov, Ph.D.

Description: The goal of this course is to cover system issues and tools for parallel processing using multiple computers or computers with multiple internal processors. In the first part of the course, we will present some general concepts of parallel machine models, processes, mutual exclusion, process synchronization, message passing, and programming languages for parallel computing and scheduling. Then we will concentrate on design and analysis of parallel algorithms. We will introduce and use Pthreads for shared memory multiprocessor systems and PVM/MPI for distributed networked computers. Students are encouraged to participate in practical projects and implement different parallel algorithms.

Text book:

• Barry Wilkinson, Allen Michael, "Parallel Programming: techniques and applications using networked workstations and parallel computers", Prentice Hall, 1999, content on: http://www.coe.uncc.edu/~abw/parallel/par_prog/PPBook.pdf

Transparencies: http://www.cs.uncc.edu/~abw/CSCI3145/
• David E. Culler, Jaswinder Pal Singh, Anoop Gupta, "Parallel Computer Architecture: A Hardware Software Approach," Morgan Kaufmann Publishers, 1999.

• Kai Hwang, "Computer Architecture: Parallelism, Scalability, Programmability" , McGraw-Hill, Inc., 1993.
• Ian Foster, "Designing and Building Parallel Programs, Concepts and Tools for Parallel Software Engineering", Addison Wesley, 1995, also available on-line http://www-unix.mcs.anl.gov/dbpp/
• Al Geist, Adam Beguelin, Jack Dongarra, Weicheng Jiang, Robert Manchek, Vaidy Sunderam, "PVM: Parallel Virtual Machine A Users' Guide and Tutorial for Networked Parallel Computing", The MIT Press, 1994, also available on-line http://www.netlib.org/pvm3/book/pvm-book.html
• Bradford Nichols, Dick Buttlar, Jackie Farrel, "Pthreads Programming", O’Reilly & Associates, Inc., 1996. http://www.ora.com/, examples:

anonymous ftp ftp.uu.net with e-mail as a password, and /published/oreilly/nutshell/pthreads/examples.tar.gz
• David A. Patterson, John L. Hennesy, "Computer Architecture A Quantitative Approach," Second Edition, Morgan Kaufmann Publishers, 1996.

Grading:

• Homeworks (2*4%) - 8%
• Laboratory Assignments (3*9%) - 27%
• Essay/Project - 10%
• Midterm exam - 25%
• Final exam - 30%

Important Dates

• Midterm Exam: Thursday October 19, 2000.
• Last day of Class: Tuesday December 5, 2000.
• Comprehensive final exam: Tuesday December 12, 2000. 6:30-9 PM

Course Outline

• Introduction (Parallel Computer Models, Applications, Problems, Principles of Scalable Performance - 6 lectures)
• Parallel Computer Architecture (System organization, Processor and Memory Hierarchies, Buses, Interconnection networks, PP issues - 4 lectures)
• OS issues (Tasks, Scheduling, Semaphores, Message passing - 2 lectures)
• Parallel program development (Parallel Models and Languages, Debugging, Synchronization - 2 lectures)
• Pthreads (5 lectures)
• PVM (3 lectures)
• MPI (3 lectures)
• Essay and Project presentations (2 lectures)

Course notes

Lab Assignments

PVM based distributed sorting due on October 26, 2000.
MPI based matrix multiplication due on November 21, 2000.
p-thread based bit vector processing due on December 4, 2000.

Homeworks

Homework#1, due on Thursday, October 5, 2000.

Homework#2, due on Tuesday, December 5, 2000.

Essays
List of available essay topics. All essays must be approved. Final version (Word document 10-15 pages) is due on November 30, 2000, @ 5:30 pm.