Specx: a C++ task-based runtime system for heterogeneous distributed architectures

TL;DR

Specx is a modern C++ runtime system enabling efficient task-based parallelization across distributed heterogeneous architectures, including CPUs and GPUs, by integrating communication within the task graph.

Contribution

It introduces Specx, a novel task-based runtime system supporting heterogeneous distributed computing with integrated communication, demonstrating its effectiveness on parallel applications.

Findings

Supports CPUs and GPUs (CUDA/HIP) simultaneously

Integrates communication into the task graph

Demonstrates potential with parallel applications

Abstract

Parallelization is needed everywhere, from laptops and mobile phones to supercomputers. Among parallel programming models, task-based programming has demonstrated a powerful potential and is widely used in high-performance scientific computing. Not only does it allow for efficient parallelization across distributed heterogeneous computing nodes, but it also allows for elegant source code structuring by describing hardware-independent algorithms. In this paper, we present Specx, a task-based runtime system written in modern C++. Specx supports distributed heterogeneous computing by simultaneously exploiting CPUs and GPUs (CUDA/HIP) and incorporating communication into the task graph. We describe the specificities of Specx and demonstrate its potential by running parallel applications.