Dynamic Content Caching with Waiting Costs via Restless Multi-Armed Bandits

TL;DR

This paper models dynamic content caching with waiting costs as a Restless Multi-Armed Bandit problem and proposes a Whittle index policy that is nearly optimal and outperforms existing strategies.

Contribution

It introduces a novel RMAB formulation for cache management with waiting costs and explicitly derives the Whittle indices for this problem.

Findings

The proposed Whittle index policy performs close to the optimal policy.

It significantly outperforms existing caching policies.

The policy is asymptotically optimal as cache size grows.

Abstract

We consider a system with a local cache connected to a backend server and an end user population. A set of contents are stored at the the server where they continuously get updated. The local cache keeps copies, potentially stale, of a subset of the contents. The users make content requests to the local cache which either can serve the local version if available or can fetch a fresh version or can wait for additional requests before fetching and serving a fresh version. Serving a stale version of a content incurs an age-of-version(AoV) dependent ageing cost, fetching it from the server incurs a fetching cost, and making a request wait incurs a per unit time waiting cost. We focus on the optimal actions subject to the cache capacity constraint at each decision epoch, aiming at minimizing the long term average cost. We pose the problem as a Restless Multi-armed Bandit(RMAB) Problem and propose a Whittle index based policy which is known to be asymptotically optimal. We explicitly characterize the Whittle indices. We numerically evaluate the proposed policy and also compare it to a greedy policy. We show that it is close to the optimal policy and substantially outperforms the exising policies.