Towards CXL Resilience to CPU Failures

TL;DR

This paper introduces ReCXL, an extension to the CXL specification that enhances resilience to node failures through replication and logging, enabling fault-tolerant shared-memory computing with minimal performance overhead.

Contribution

ReCXL extends the CXL standard to support fault tolerance against node failures by implementing replication and logging mechanisms for recovery.

Findings

ReCXL achieves fault-tolerance with only 30% slowdown.

The system effectively recovers application state after node failures.

ReCXL maintains data consistency through replication and logging.

Abstract

Compute Express Link (CXL) 3.0 and beyond allows the compute nodes of a cluster to share data with hardware cache coherence and at the granularity of a cache line. This enables shared-memory semantics for distributed computing, but introduces new resilience challenges: a node failure leads to the loss of the dirty data in its caches, corrupting application state. Unfortunately, the CXL specification does not consider processor failures. Moreover, when a component fails, the specification tries to isolate it and continue application execution; there is no attempt to bring the application to a consistent state. To address these limitations, this paper extends the CXL specification to be resilient to node failures, and to correctly recover the application after node failures. We call the system ReCXL. To handle the failure of nodes, ReCXL augments the coherence transaction of a write with messages that propagate the update to a small set of other nodes (i.e., Replicas). Replicas save the update in a hardware Logging Unit. Such replication ensures resilience to node failures. Then, at regular intervals, the Logging Units dump the updates to memory. Recovery involves using the logs in the Logging Units to bring the directory and memory to a correct state. Our evaluation shows that ReCXL enables fault-tolerant execution with only a 30% slowdown over the same platform with no fault-tolerance support.