Fault Tolerance for Remote Memory Access Programming Models

TL;DR

This paper develops scalable fault tolerance mechanisms for Remote Memory Access (RMA) programming models, addressing a key gap in resilience strategies for high-performance computing systems.

Contribution

It introduces a novel fault tolerance model for RMA, designing efficient in-memory checkpointing, logging, and recovery schemes suitable for future exascale architectures.

Findings

In-memory checkpointing and logging achieve high resilience with minimal overhead.

The proposed protocols are scalable and effective for both flat and hierarchical hardware.

The fault-tolerance schemes are robust even with decreasing memory per core in exascale systems.

Abstract

Remote Memory Access (RMA) is an emerging mechanism for programming high-performance computers and datacenters. However, little work exists on resilience schemes for RMA-based applications and systems. In this paper we analyze fault tolerance for RMA and show that it is fundamentally different from resilience mechanisms targeting the message passing (MP) model. We design a model for reasoning about fault tolerance for RMA, addressing both flat and hierarchical hardware. We use this model to construct several highly-scalable mechanisms that provide efficient low-overhead in-memory checkpointing, transparent logging of remote memory accesses, and a scheme for transparent recovery of failed processes. Our protocols take into account diminishing amounts of memory per core, one of major features of future exascale machines. The implementation of our fault-tolerance scheme entails negligible additional overheads. Our reliability model shows that in-memory checkpointing and logging provide high resilience. This study enables highly-scalable resilience mechanisms for RMA and fills a research gap between fault tolerance and emerging RMA programming models.