Building Atomic, Crash-Consistent Data Stores with Disaggregated Persistent Memory

TL;DR

This paper investigates architectures for disaggregated persistent memory (DPM) in datacenters, proposing three designs and evaluating their performance, scalability, and CPU costs for building atomic, crash-consistent data stores.

Contribution

It introduces three novel DPM architectures and demonstrates their implementation and performance characteristics for building crash-consistent data stores.

Findings

DPM-Direct excels in small read operations.

DPM-Central offers best small-scale write performance.

DPM-Sep provides balanced overall performance.

Abstract

Byte-addressable persistent memories (PM) has finally made their way into production. An important and pressing problem that follows is how to deploy them in existing datacenters. One viable approach is to attach PM as self-contained devices to the network as disaggregated persistent memory, or DPM. DPM requires no changes to existing servers in datacenters; without the need to include a processor, DPM devices are cheap to build; and by sharing DPM across compute servers, they offer great elasticity and efficient resource packing. This paper explores different ways to organize DPM and to build data stores with DPM. Specifically, we propose three architectures of DPM: 1) compute nodes directly access DPM (DPM-Direct); 2) compute nodes send requests to a coordinator server, which then accesses DPM to complete a request (DPM-Central); and 3) compute nodes directly access DPM for data operations and communicate with a global metadata server for the control plane (DPM-Sep). Based on these architectures, we built three atomic, crash-consistent data stores. We evaluated their performance, scalability, and CPU cost with micro-benchmarks and YCSB. Our evaluation results show that DPM-Direct has great small-size read but poor write performance; DPM-Central has the best write performance when the scale of the cluster is small but performs poorly when the scale increases; and DPM-Sep performs well overall.