Towards a Scalable and Efficient PGAS-based Distributed OpenMP

TL;DR

This paper introduces DiOMP, a PGAS-based extension to OpenMP that simplifies distributed memory programming while achieving better performance and scalability than traditional MPI+OpenMP models.

Contribution

It presents a novel PGAS-integrated OpenMP model using LLVM and GASNet-EX, improving ease of use and performance for distributed memory systems.

Findings

DiOMP achieves up to 25% higher bandwidth than MPI+OpenMP.

DiOMP reduces latency by up to 45% compared to MPI+OpenMP.

Performance evaluated on Ookami and NERSC Perlmutter platforms.

Abstract

MPI+X has been the de facto standard for distributed memory parallel programming. It is widely used primarily as an explicit two-sided communication model, which often leads to complex and error-prone code. Alternatively, PGAS model utilizes efficient one-sided communication and more intuitive communication primitives. In this paper, we present a novel approach that integrates PGAS concepts into the OpenMP programming model, leveraging the LLVM compiler infrastructure and the GASNet-EX communication library. Our model addresses the complexity associated with traditional MPI+OpenMP programming models while ensuring excellent performance and scalability. We evaluate our approach using a set of micro-benchmarks and application kernels on two distinct platforms: Ookami from Stony Brook University and NERSC Perlmutter. The results demonstrate that DiOMP achieves superior bandwidth and lower latency compared to MPI+OpenMP, up to 25% higher bandwidth and down to 45% on latency. DiOMP offers a promising alternative to the traditional MPI+OpenMP hybrid programming model, towards providing a more productive and efficient way to develop high-performance parallel applications for distributed memory systems.