Reinforcenment Learning-Aided NOMA Random Access: An AoI-Based Timeliness Perspective

TL;DR

This paper proposes a reinforcement learning-based NOMA random access scheme that optimizes the age-of-information in IoT networks, achieving significant AoI improvements with minimal feedback overhead.

Contribution

It introduces a novel AoI-aware Q-learning approach for NOMA random access, optimizing transmission timing and power levels in a distributed manner.

Findings

Significant reduction in average AoI compared to existing schemes.

Achieves near-optimal AoI performance with only one feedback bit per slot.

Maintains high network throughput despite AoI optimization.

Abstract

In this paper, we investigate the age-of-information (AoI) of a power domain non-orthogonal multiple access (NOMA) network, where multiple internet-of-things (IoT) devices transmit to a common gateway in a grant-free random fashion. More specifically, we consider a framed setup composed of multiple time slots, and resort to the $Q$-learning algorithm to properly define, in a distributed manner, the time slot and the power level each IoT device transmits within a frame. In the proposed AoI-QL-NOMA scheme, the $Q$-learning reward is adapted with the aim of minimizing the average AoI of the network, while only requiring a single feedback bit per time slot, in a frame basis. Our results show that AoI-QL-NOMA significantly improves the AoI performance compared to some recently proposed schemes, without significantly reducing the network throughput.