A Reinforcement Learning Approach to Age of Information in Multi-User Networks

TL;DR

This paper develops a reinforcement learning-based scheduling strategy to minimize the average age of information in multi-user networks, effectively handling unknown channel dynamics without prior statistical knowledge.

Contribution

It introduces an RL approach for AoI optimization in multi-user networks, extending beyond traditional methods that require known channel models.

Findings

RL methods outperform traditional policies in unknown channel conditions

The proposed RL approach effectively reduces AoI in simulations

Different RL algorithms are compared for efficiency and performance

Abstract

Scheduling the transmission of time-sensitive data to multiple users over error-prone communication channels is studied with the goal of minimizing the long-term average age of information (AoI) at the users under a constraint on the average number of transmissions at the source node. After each transmission, the source receives an instantaneous ACK/NACK feedback from the intended receiver and decides on what time and to which user to transmit the next update. The optimal scheduling policy is first studied under different feedback mechanisms when the channel statistics are known; in particular, the standard automatic repeat request (ARQ) and hybrid ARQ (HARQ) protocols are considered. Then a reinforcement learning (RL) approach is introduced, which does not assume any a priori information on the random processes governing the channel states. Different RL methods are verified and compared through numerical simulations.