A Survey of Freshness-Aware Wireless Networking with Reinforcement Learning

TL;DR

This survey reviews reinforcement learning approaches for optimizing data freshness in wireless networks, categorizing methods by decision type and addressing challenges in next-generation systems.

Contribution

It provides a unified taxonomy and framework for applying RL to freshness-aware wireless networking, covering various decision types and identifying open research challenges.

Findings

Organized AoI and freshness models into native, function-based, and application-oriented categories.

Proposed a policy-centric taxonomy for RL decisions impacting data freshness.

Highlighted open challenges like delayed decisions and cross-layer design in RL-based freshness optimization.

Abstract

The age of information (AoI) has become a central measure of data freshness in modern wireless systems, yet existing surveys either focus on classical AoI formulations or provide broad discussions of reinforcement learning (RL) in wireless networks without addressing freshness as a unified learning problem. Motivated by this gap, this survey examines RL specifically through the lens of AoI and generalized freshness optimization. We organize AoI and its variants into native, function-based, and application-oriented families, providing a clearer view of how freshness should be modeled in B5G and 6G systems. Building on this foundation, we introduce a policy-centric taxonomy that reflects the decisions most relevant to freshness, consisting of update-control RL, medium-access RL, risk-sensitive RL, and multi-agent RL. This structure provides a coherent framework for understanding how learning can support sampling, scheduling, trajectory planning, medium access, and distributed coordination. We further synthesize recent progress in RL-driven freshness control and highlight open challenges related to delayed decision processes, stochastic variability, and cross-layer design. The goal is to establish a unified foundation for learning-based freshness optimization in next-generation wireless networks.