Lectures on Parallel Computing
Jesper Larsson Tr\"aff

TL;DR
This paper provides comprehensive lecture notes on fundamental parallel computing concepts, frameworks like OpenMP, MPI, pthreads, and practical guidance for students and educators in high-performance and shared-memory parallel programming.
Contribution
It offers detailed explanations, code examples, and exercises focusing on deterministic parallel algorithms and principles, serving as an educational resource for undergraduate and graduate courses.
Findings
C code examples illustrating parallel algorithms
Guidelines for efficient parallel program design
Exercises for self-study and implementation
Abstract
These lecture notes are designed to accompany an imaginary, virtual, undergraduate, one or two semester course on fundamentals of Parallel Computing as well as to serve as background and reference for graduate courses on High-Performance Computing, parallel algorithms and shared-memory multiprocessor programming. They introduce theoretical concepts and tools for expressing, analyzing and judging parallel algorithms and, in detail, cover the two most widely used concrete frameworks OpenMP and MPI as well as the threading interface pthreads for writing parallel programs for either shared or distributed memory parallel computers with emphasis on general concepts and principles. Code examples are given in a C-like style and many are actual, correct C code. The lecture notes deliberately do not cover GPU architectures and GPU programming, but the general concerns, guidelines and principles…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsParallel Computing and Optimization Techniques · Distributed and Parallel Computing Systems · Cloud Computing and Resource Management
