Asymptotic Miss Ratio of LRU Caching with Consistent Hashing

TL;DR

This paper derives the asymptotic miss ratio for a cluster of LRU caches organized by consistent hashing, showing they behave like a single pooled cache under large cache sizes, aiding scalable data caching system design.

Contribution

It provides the first theoretical characterization of the miss ratio for LRU cache clusters with consistent hashing, establishing a virtual cache equivalence under large cache sizes.

Findings

Caches can be modeled as a single virtual cache for large sizes.

Theoretical framework applies to real-world big data caching systems.

Provides insights into cache performance in distributed systems.

Abstract

To efficiently scale data caching infrastructure to support emerging big data applications, many caching systems rely on consistent hashing to group a large number of servers to form a cooperative cluster. These servers are organized together according to a random hash function. They jointly provide a unified but distributed hash table to serve swift and voluminous data item requests. Different from the single least-recently-used (LRU) server that has already been extensively studied, theoretically characterizing a cluster that consists of multiple LRU servers remains yet to be explored. These servers are not simply added together; the random hashing complicates the behavior. To this end, we derive the asymptotic miss ratio of data item requests on a LRU cluster with consistent hashing. We show that these individual cache spaces on different servers can be effectively viewed as if they could be pooled together to form a single virtual LRU cache space parametrized by an appropriate cache size. This equivalence can be established rigorously under the condition that the cache sizes of the individual servers are large. For typical data caching systems this condition is common. Our theoretical framework provides a convenient abstraction that can directly apply the results from the simpler single LRU cache to the more complex LRU cluster with consistent hashing.