Examining MPI and its Extensions for Asynchronous Multithreaded Communication

TL;DR

This paper evaluates MPI extensions for asynchronous multithreaded communication within AMT systems, revealing performance gains but also highlighting limitations in scalability and efficiency that need addressing.

Contribution

It provides an empirical assessment of MPI's Virtual Communication Interface and Continuation extensions in AMT environments, identifying key areas for improvement.

Findings

Extensions can improve performance over standard MPI

Current continuation limits multithreaded message rates

One-VCI-per-thread mode is ineffective in real systems

Abstract

The increasing complexity of HPC architectures and the growing adoption of irregular scientific algorithms demand efficient support for asynchronous, multithreaded communication. This need is especially pronounced with Asynchronous Many-Task (AMT) systems. This communication pattern was not a consideration during the design of the original MPI specification. The MPI community has recently introduced several extensions to address these evolving requirements. This work evaluates two such extensions, the Virtual Communication Interface (VCI) and the Continuation extensions, in the context of an established AMT runtime HPX. We begin by using an MPI-level microbenchmark, modeled from HPX's low-level communication mechanism, to measure the peak performance potential of these extensions. We then integrate them into HPX to evaluate their effectiveness in real-world scenarios. Our results show that while these extensions can enhance performance compared to standard MPI, areas for improvement remain. The current continuation proposal limits the maximum multithreaded message rate achievable in the multi-VCI setting. Furthermore, the recommended one-VCI-per-thread mode proves ineffective in real-world systems due to the attentiveness problem. These findings underscore the importance of improving intra-VCI threading efficiency to achieve scalable multithreaded communication and fully realize the benefits of recent MPI extensions.