Exploiting Deep Reinforcement Learning for Edge Caching in Cell-Free Massive MIMO Systems

TL;DR

This paper introduces a deep reinforcement learning approach using soft actor-critic for proactive content caching in cell-free massive MIMO systems to enhance QoE and reduce delay in high-speed railway communications.

Contribution

It presents a novel DRL-based cache placement algorithm that outperforms heuristic methods and benchmarks in dynamic wireless environments.

Findings

DRL with SAC improves QoE over heuristic methods.

Proactive caching reduces end-to-end delay.

SAC accurately predicts user requests in high-mobility scenarios.

Abstract

Cell-free massive multiple-input-multiple-output is promising to meet the stringent quality-of-experience (QoE) requirements of railway wireless communications by coordinating many successional access points (APs) to serve the onboard users coherently. A key challenge is how to deliver the desired contents timely due to the radical changing propagation environment caused by the growing train speed. In this paper, we propose to proactively cache the likely-requesting contents at the upcoming APs which perform the coherent transmission to reduce end-to-end delay. A long-term QoE-maximization problem is formulated and two cache placement algorithms are proposed. One is based on heuristic convex optimization (HCO) and the other exploits deep reinforcement learning (DRL) with soft actor-critic (SAC). Compared to the conventional benchmark, numerical results show the advantage of our proposed algorithms on QoE and hit probability. With the advanced DRL model, SAC outperforms HCO on QoE by predicting the user requests accurately.