Transformer Actor-Critic for Efficient Freshness-Aware Resource Allocation

TL;DR

This paper introduces a Transformer-enhanced deep reinforcement learning approach for minimizing age of information in multi-user NOMA wireless networks, effectively prioritizing critical users and improving resource allocation efficiency.

Contribution

It presents a novel Transformer-based DRL framework for AoI minimization that captures user dependencies and priorities in NOMA systems, enhancing scalability and performance.

Findings

Reduces average AoI compared to baseline methods

Learns to prioritize high-importance users over training

Attention maps show meaningful focus patterns aligned with user priorities

Abstract

Emerging applications such as autonomous driving and industrial automation demand ultra-reliable and low-latency communication (URLLC), where maintaining fresh and timely information is critical. A key performance metric in such systems is the age of information (AoI). This paper addresses AoI minimization in a multi-user uplink wireless network using non-orthogonal multiple access (NOMA), where users offload tasks to a base station. The system must handle user heterogeneity in task sizes, AoI thresholds, and penalty sensitivities, while adhering to NOMA constraints on user scheduling. We propose a deep reinforcement learning (DRL) framework based on proximal policy optimization (PPO), enhanced with a Transformer encoder. The attention mechanism allows the agent to focus on critical user states and capture inter-user dependencies, improving policy performance and scalability. Extensive simulations show that our method reduces average AoI compared to baselines. We also analyze the evolution of attention weights during training and observe that the model progressively learns to prioritize high-importance users. Attention maps reveal meaningful structure: early-stage policies exhibit uniform attention, while later stages show focused patterns aligned with user priority and NOMA constraints. These results highlight the promise of attention-driven DRL for intelligent, priority-aware resource allocation in next-generation wireless systems.