Benchmarking Usability and Performance of Multicore Languages

TL;DR

This paper compares four different multicore programming languages through implementing benchmarks, evaluating usability and performance, and highlighting strengths, weaknesses, and the importance of expert knowledge in parallel programming.

Contribution

It provides a comprehensive experimental comparison of four diverse parallel programming approaches, including expert reviews, to inform language selection.

Findings

All approaches have distinct strengths and weaknesses.

Expert review emphasizes the importance of expert knowledge.

Performance varies significantly across languages and benchmarks.

Abstract

Developers face a wide choice of programming languages and libraries supporting multicore computing. Ever more diverse paradigms for expressing parallelism and synchronization become available while their influence on usability and performance remains largely unclear. This paper describes an experiment comparing four markedly different approaches to parallel programming: Chapel, Cilk, Go, and Threading Building Blocks (TBB). Each language is used to implement sequential and parallel versions of six benchmark programs. The implementations are then reviewed by notable experts in the language, thereby obtaining reference versions for each language and benchmark. The resulting pool of 96 implementations is used to compare the languages with respect to source code size, coding time, execution time, and speedup. The experiment uncovers strengths and weaknesses in all approaches, facilitating an informed selection of a language under a particular set of requirements. The expert review step furthermore highlights the importance of expert knowledge when using modern parallel programming approaches.