Ditto: An Elastic and Adaptive Memory-Disaggregated Caching System

TL;DR

Ditto is a novel elastic caching system designed for disaggregated memory architectures, enabling flexible resource allocation and adaptive caching algorithms to improve cache hit rates in cloud services.

Contribution

It introduces the first caching system on disaggregated memory that efficiently executes algorithms and adaptively switches between them based on performance.

Findings

Outperforms state-of-the-art caching systems by up to 3.6x in real workloads.

Achieves up to 9x improvement in YCSB benchmarks.

Effectively adapts to resource changes on disaggregated memory architectures.

Abstract

In-memory caching systems are fundamental building blocks in cloud services. However, due to the coupled CPU and memory on monolithic servers, existing caching systems cannot elastically adjust resources in a resource-efficient and agile manner. To achieve better elasticity, we propose to port in-memory caching systems to the disaggregated memory (DM) architecture, where compute and memory resources are decoupled and can be allocated flexibly. However, constructing an elastic caching system on DM is challenging since accessing cached objects with CPU-bypass remote memory accesses hinders the execution of caching algorithms. Moreover, the elastic changes of compute and memory resources on DM affect the access patterns of cached data, compromising the hit rates of caching algorithms. We design Ditto, the first caching system on DM, to address these challenges. Ditto first proposes a client-centric caching framework to efficiently execute various caching algorithms in the compute pool of DM, relying only on remote memory accesses. Then, Ditto employs a distributed adaptive caching scheme that adaptively switches to the best-fit caching algorithm in real-time based on the performance of multiple caching algorithms to improve cache hit rates. Our experiments show that Ditto effectively adapts to the changing resources on DM and outperforms the state-of-the-art caching systems by up to 3.6x in real-world workloads and 9x in YCSB