Telepathic Datacenters: Fast RPCs using Shared CXL Memory

TL;DR

RPCool leverages CXL's shared memory to significantly reduce RPC latency in datacenters by avoiding serialization and efficiently managing shared data, with fallback to RDMA for scalability.

Contribution

Introduces RPCool, a novel framework that exploits CXL shared memory for fast RPCs, combining pointer sharing with safety mechanisms and RDMA fallback for scalability.

Findings

RPCool reduces round-trip latency by up to 7.2× compared to existing methods.

RPCool performs comparably or better across various workloads.

Effective memory permission management ensures safety in shared memory RPCs.

Abstract

Datacenter applications often rely on remote procedure calls (RPCs) for fast, efficient, and secure communication. However, RPCs are slow, inefficient, and hard to use as they require expensive serialization and compression to communicate over a packetized serial network link. Compute Express Link 3.0 (CXL) offers an alternative solution, allowing applications to share data using a cache-coherent, shared-memory interface across clusters of machines. RPCool is a new framework that exploits CXL's shared memory capabilities. RPCool avoids serialization by passing pointers to data structures in shared memory. While avoiding serialization is useful, directly sharing pointer-rich data eliminates the isolation that copying data over traditional networks provides, leaving the receiver vulnerable to invalid pointers and concurrent updates to shared data by the sender. RPCool restores this safety with careful and efficient management of memory permissions. Another significant challenge with CXL shared memory capabilities is that they are unlikely to scale to an entire datacenter. RPCool addresses this by falling back to RDMA-based communication. Overall, RPCool reduces the round-trip latency by 1.93$\times$ and 7.2$\times$ compared to state-of-the-art RDMA and CXL-based RPC mechanisms, respectively. Moreover, RPCool performs either comparably or better than other RPC mechanisms across a range of workloads.