Chunks and Tasks: a programming model for parallelization of dynamic algorithms

TL;DR

The paper introduces Chunks and Tasks, a parallel programming model that simplifies dynamic algorithm parallelization by abstracting data and work distribution, aiming for high performance and user-friendliness.

Contribution

It presents a novel programming model with abstractions for data and work, along with a C++ library implementation for efficient parallel execution.

Findings

Demonstrated performance on sparse matrix multiplication

Achieved high efficiency with dynamic data and work distribution

Facilitated development of complex parallel applications

Abstract

We propose Chunks and Tasks, a parallel programming model built on abstractions for both data and work. The application programmer specifies how data and work can be split into smaller pieces, chunks and tasks, respectively. The Chunks and Tasks library maps the chunks and tasks to physical resources. In this way we seek to combine user friendliness with high performance. An application programmer can express a parallel algorithm using a few simple building blocks, defining data and work objects and their relationships. No explicit communication calls are needed; the distribution of both work and data is handled by the Chunks and Tasks library. This makes efficient implementation of complex applications that require dynamic distribution of work and data easier. At the same time, Chunks and Tasks imposes restrictions on data access and task dependencies that facilitates the development of high performance parallel back ends. We discuss the fundamental abstractions underlying the programming model, as well as performance and fault resilience considerations. We also present a pilot C++ library implementation for clusters of multicore machines and demonstrate its performance for sparse blocked matrix-matrix multiplication.