Coded Caching via Federated Deep Reinforcement Learning in Fog Radio Access Networks

TL;DR

This paper proposes a federated deep reinforcement learning approach for coded caching in fog radio access networks, effectively adapting to content popularity changes to reduce access delay.

Contribution

It introduces a novel federated deep reinforcement learning framework for coded caching that accounts for dynamic content popularity in F-RANs.

Findings

Outperforms benchmarks in reducing content access delay.

Maintains stable performance despite popularity fluctuations.

Balances local caching and global multicasting gains.

Abstract

In this paper, the placement strategy design of coded caching in fog-radio access networks (F-RANs) is investigated. By considering time-variant content popularity, federated deep reinforcement learning is exploited to learn the placement strategy for our coded caching scheme. Initially, the placement problem is modeled as a Markov decision process (MDP) to capture the popularity variations and minimize the long-term content access delay. The reformulated sequential decision problem is solved by dueling double deep Q-learning (dueling DDQL). Then, federated learning is applied to learn the relatively low-dimensional local decision models and aggregate the global decision model, which alleviates over-consumption of bandwidth resources and avoids direct learning of a complex coded caching decision model with high-dimensional state space. Simulation results show that our proposed scheme outperforms the benchmarks in reducing the content access delay, keeping the performance stable, and trading off between the local caching gain and the global multicasting gain.