Clio: A Hardware-Software Co-Designed Disaggregated Memory System

TL;DR

Clio is a hardware-software co-designed disaggregated memory system that offers high throughput, low latency, scalability, and energy efficiency by integrating processing power at memory nodes.

Contribution

We introduce Clio, a novel hardware-based disaggregated memory system with integrated processing, co-designed with OS, hardware, and network components for improved performance and management.

Findings

Achieves 100 Gbps throughput per memory node

End-to-end latency of 2.5 microseconds median

Outperforms RDMA and SmartNIC-based systems in scalability and energy efficiency

Abstract

Memory disaggregation has attracted great attention recently because of its benefits in efficient memory utilization and ease of management. So far, memory disaggregation research has all taken one of two approaches: building/emulating memory nodes using regular servers or building them using raw memory devices with no processing power. The former incurs higher monetary cost and faces tail latency and scalability limitations, while the latter introduces performance, security, and management problems. Server-based memory nodes and memory nodes with no processing power are two extreme approaches. We seek a sweet spot in the middle by proposing a hardware-based memory disaggregation solution that has the right amount of processing power at memory nodes. Furthermore, we take a clean-slate approach by starting from the requirements of memory disaggregation and designing a memory-disaggregation-native system. We built Clio, a disaggregated memory system that virtualizes, protects, and manages disaggregated memory at hardware-based memory nodes. The Clio hardware includes a new virtual memory system, a customized network system, and a framework for computation offloading. In building Clio, we not only co-design OS functionalities, hardware architecture, and the network system, but also co-design compute nodes and memory nodes. Our FPGA prototype of Clio demonstrates that each memory node can achieve 100 Gbps throughput and an end-to-end latency of 2.5 us at median and 3.2us at the 99th percentile. Clio also scales much better and has orders of magnitude lower tail latency than RDMA. It has 1.1x to 3.4x energy saving compared to CPU-based and SmartNIC-based disaggregated memory systems and is 2.7x faster than software-based SmartNIC solutions.