Proactive Caching for Energy-Efficiency in Wireless Networks: A Markov Decision Process Approach

TL;DR

This paper introduces a Markov Decision Process-based method for proactive caching in wireless networks, optimizing energy efficiency by decoupling complex dependencies and reducing computational complexity with a novel structural approach.

Contribution

It presents a new MDP model for joint pushing and caching, along with a low-dimensional decomposition and a non-iterative algorithm exploiting generalized monotonicity.

Findings

Achieves near-optimal performance close to theoretical bounds.

Significantly reduces computational complexity.

Outperforms unadapted MDP solutions in efficiency.

Abstract

Content caching in wireless networks provides a substantial opportunity to trade off low cost memory storage with energy consumption, yet finding the optimal causal policy with low computational complexity remains a challenge. This paper models the Joint Pushing and Caching (JPC) problem as a Markov Decision Process (MDP) and provides a solution to determine the optimal randomized policy. A novel approach to decouple the influence from buffer occupancy and user requests is proposed to turn the high-dimensional optimization problem into three low-dimensional ones. Furthermore, a non-iterative algorithm to solve one of the sub-problems is presented, exploiting a structural property we found as \textit{generalized monotonicity}, and hence significantly reduces the computational complexity. The result attains close performance in comparison with theoretical bounds from non-practical policies, while benefiting from higher time efficiency than the unadapted MDP solution.