Lower Bounds for Caching with Delayed Hits

TL;DR

This paper investigates the theoretical limits of caching with delayed hits, establishing lower bounds on algorithm performance and demonstrating that the latency in such models is not inherently antimonotone, with a proposed modification to address this.

Contribution

It provides the first lower bound on the competitive ratio for delayed hits caching algorithms and introduces a modified model ensuring latency is antimonotone.

Findings

The competitive ratio of any deterministic algorithm is at least Omega(kZ).

Latency in the delayed hits model can increase with cache hits, violating antimonotonicity.

A modified model is proposed that guarantees antimonotonicity of latency.

Abstract

Caches are a fundamental component of latency-sensitive computer systems. Recent work of [ASWB20] has initiated the study of delayed hits: a phenomenon in caches that occurs when the latency between the cache and backing store is much larger than the time between new requests. We present two results for the delayed hits caching model. (1) Competitive ratio lower bound. We prove that the competitive ratio of the algorithm in [ASWB20], and more generally of any deterministic online algorithm for delayed hits, is at least Omega(kZ), where k is the cache size and Z is the delay parameter. (2) Antimonotonicity of the delayed hits latency. Antimonotonicity is a naturally desirable property of cache latency: having a cache hit instead of a cache miss should result in lower overall latency. We prove that the latency of the delayed hits model is not antimonotone by exhibiting a scenario where having a cache hit instead of a miss results in an increase in overall latency. We additionally present a modification of the delayed hits model that makes the latency antimonotone.