MPIX Stream: An Explicit Solution to Hybrid MPI+X Programming

TL;DR

This paper introduces MPIX stream, a new MPI concept that explicitly passes thread and GPU stream contexts to MPI, enabling better performance optimization in heterogeneous high-performance computing systems.

Contribution

The paper proposes MPIX stream, an explicit MPI extension for passing thread and GPU stream contexts, addressing limitations of the current MPI standard.

Findings

MPIX stream enables direct mapping of execution contexts to network endpoints.

Passing GPU streams into MPI reduces CPU/GPU synchronization costs.

MPIX stream improves performance optimization in heterogeneous systems.

Abstract

The hybrid MPI+X programming paradigm, where X refers to threads or GPUs, has gained prominence in the high-performance computing arena. This corresponds to a trend of system architectures growing more heterogeneous. The current MPI standard only specifies the compatibility levels between MPI and threading runtimes. No MPI concept or interface exists for applications to pass thread context or GPU stream context to MPI implementations explicitly. This lack has made performance optimization complicated in some cases and impossible in other cases. We propose a new concept in MPI, called MPIX stream, to represent the general serial execution context that exists in X runtimes. MPIX streams can be directly mapped to threads or GPU execution streams. Passing thread context into MPI allows implementations to precisely map the execution contexts to network endpoints. Passing GPU execution context into MPI allows implementations to directly operate on GPU streams, lowering the CPU/GPU synchronization cost.