Optimized Dynamic Cache Instantiation and Accurate LRU Approximations under Time-varying Request Volume

TL;DR

This paper explores dynamic cache instantiation in cloud environments, providing models and approximations that optimize cache costs and performance under fluctuating request rates, with significant potential savings.

Contribution

It introduces novel analytic models for time-varying cache workloads, including bounds and approximations for LRU caches, and evaluates cost-performance tradeoffs of dynamic cache deployment.

Findings

Dynamic cache instantiation yields substantial cost savings.

Savings depend heavily on object popularity skew.

Selective cache insertion enhances benefits over traditional edge caches.

Abstract

Content-delivery applications can achieve scalability and reduce wide-area network traffic using geographically distributed caches. However, each deployed cache has an associated cost, and under time-varying request rates (e.g., a daily cycle) there may be long periods when the request rate from the local region is not high enough to justify this cost. Cloud computing offers a solution to problems of this kind, by supporting dynamic allocation and release of resources. In this paper, we analyze the potential benefits from dynamically instantiating caches using resources from cloud service providers. We develop novel analytic caching models that accommodate time-varying request rates, transient behavior as a cache fills following instantiation, and selective cache insertion policies. Within the context of a simple cost model, we then develop bounds and compare policies with optimized parameter selections to obtain insights into key cost/performance tradeoffs. We find that dynamic cache instantiation can provide substantial cost reductions, that potential reductions strongly dependent on the object popularity skew, and that selective cache insertion can be even more beneficial in this context than with conventional edge caches. Finally, our contributions also include accurate and easy-to-compute approximations that are shown applicable to LRU caches under time-varying workloads.