History-Aware Online Cache Placement in Fog-Assisted IoT Systems: An Integration of Learning and Control

TL;DR

This paper introduces CPHBL, a novel cache placement scheme for fog-assisted IoT systems that combines online learning, control, and historical data to minimize regret and manage storage costs effectively.

Contribution

It formulates cache placement as a constrained CMAB problem and develops a history-aware bandit learning method integrated with control techniques.

Findings

CPHBL achieves sublinear regret bounds.

CPHBL outperforms deep reinforcement learning approaches.

Effective management of storage costs under constraints.

Abstract

In Fog-assisted IoT systems, it is a common practice to cache popular content at the network edge to achieve high quality of service. Due to uncertainties in practice such as unknown file popularities, cache placement scheme design is still an open problem with unresolved challenges: 1) how to maintain time-averaged storage costs under budgets, 2) how to incorporate online learning to aid cache placement to minimize performance loss (a.k.a. regret), and 3) how to exploit offline historical information to further reduce regret. In this paper, we formulate the cache placement problem with unknown file popularities as a constrained combinatorial multi-armed bandit (CMAB) problem. To solve the problem, we employ virtual queue techniques to manage time-averaged storage cost constraints, and adopt history-aware bandit learning methods to integrate offline historical information into the online learning procedure to handle the exploration-exploitation tradeoff. With an effective combination of online control and history-aware online learning, we devise a Cache Placement scheme with History-aware Bandit Learning called CPHBL. Our theoretical analysis and simulations show that CPHBL achieves a sublinear time-averaged regret bound. Moreover, the simulation results verify CPHBL's advantage over the deep reinforcement learning based approach.