A Survey on Parallelism and Determinism

TL;DR

This survey reviews programming languages and models that ensure deterministic parallelism, focusing on preventing non-determinism sources like data races to improve reliability and predictability.

Contribution

It provides a comprehensive comparison of existing solutions for deterministic parallel programming and analyzes their strengths and limitations.

Findings

Deterministic languages help prevent data races and non-determinism.

Programming models vary in their ability to ensure deterministic execution.

Trade-offs exist between flexibility and guarantees of determinism.

Abstract

Parallelism is often required for performance. In these situations an excess of non-determinism is harmful as it means the program can have several different behaviours or even different results. Even in domains such as high-performance computing where parallelism is crucial for performance, the computed value should be deterministic. Unfortunately, non-determinism in programs also allows dynamic scheduling of tasks, reacting to the first task that succeeds, cancelling tasks that cannot lead to a result, etc. Non-determinism is thus both a desired asset or an undesired property depending on the situation. In practice, it is often necessary to limit non-determinism and to identify precisely the sources of non-determinism in order to control what parts of a program are deterministic or not. This survey takes the perspective of programming languages, and studies how programming models can ensure the determinism of parallel programs. This survey studies not only deterministic languages but also programming models that prevent one particularly demanding source of non-determinism: data races. Our objective is to compare existing solutions to the following questions: How programming languages can help programmers write programs that run in a parallel manner without visible non-determinism? What programming paradigms ensure this kind of properties? We study these questions and discuss the merits and limitations of different approaches.