Hierarchical Cooperative MARL for Joint Downlink PRB and Power Allocation in a 5G System

TL;DR

This paper introduces a hierarchical multi-agent reinforcement learning framework for joint downlink PRB and power allocation in 5G systems, improving throughput and fairness through staged training and cross-layer optimization.

Contribution

It presents a novel hierarchical cooperative MARL approach with staged curriculum training for efficient joint resource management in 5G, integrating system-level simulation and mobility-aware channel modeling.

Findings

Learned resource allocation components outperform traditional PF scheduling.

Full joint PRB and power control yields significant throughput gains.

Framework maintains acceptable fairness while enhancing cell throughput.

Abstract

Efficient downlink radio resource management in 5G requires jointly optimizing user scheduling and transmit-power allocation under time-varying wireless conditions. This is challenging in OFDMA systems because PRB assignment is combinatorial, power allocation is continuous, and performance depends on channel evolution, link adaptation, and long-term fairness. We propose a hierarchical cooperative multi-agent reinforcement learning framework with staged curriculum training for joint downlink PRB and power allocation in a physically grounded 5G environment. System-level simulation is implemented in Sionna, while Sionna RT supports wireless scene construction and mobility-aware ray-traced channel generation. The control task is decomposed into two sequential stages: a PRB agent learns user-level resource shares, which are converted to exact PRB assignments by a deterministic channel-aware quota resolver, and a power agent distributes the base-station power budget across users and their assigned PRB-symbol resources. The framework operates in a cross-layer loop with adaptive modulation and coding, HARQ feedback, outer-loop link adaptation, and a fairness-aware reward based on smoothed throughput and Jain's fairness index. Training stability is improved through a three-phase curriculum for PRB allocation, power control, and joint fine-tuning. Under matched channel realizations, we compare against a PF scheduler with equal-power transmission and two ablations isolating the learned PRB and power-control components. Results show that both learned components improve throughput distribution relative to PF, while the full PRB and power controller achieves the largest cell-throughput gain with only a modest reduction in Jain's fairness index.