Experiences with task-based programming using cluster nodes as OpenMP devices

TL;DR

This paper explores using cluster nodes as OpenMP devices to simplify distributed system programming, demonstrating that with minimal communication overhead, significant speedups are achievable in task-based applications.

Contribution

It introduces a novel approach to treat cluster nodes as OpenMP devices, enabling easier programming and adaptation of existing task-based applications for distributed systems.

Findings

Significant speedup achievable with minimal communication overheads

Necessary modifications enable existing applications to exploit cluster as OpenMP devices

Performance depends on communication overheads in task execution

Abstract

Programming a distributed system, such as a cluster, requires extended use of low-level communication libraries and can often become cumbersome and error prone for the average developer. In this work, we consider each node of a cluster as a separate OpenMP device, able to run code with OpenMP directives in parallel. We make use of the OpenMP device model to provide an easy and intuitive way to program available cluster nodes. Based on that, we examine modifications that were necessary to make existing task-based applications able to exploit such an infrastructure. Finally, we evaluate the performance of the system and conclude that one can gain significant speedup, as long as the application tasks do not produce excessive communication overheads.