An Offline Multi-Agent Reinforcement Learning Framework for Radio Resource Management

TL;DR

This paper introduces an offline multi-agent reinforcement learning framework for radio resource management that improves scheduling policies for multiple access points, enhancing network performance while reducing online interaction costs.

Contribution

It proposes an offline MARL algorithm tailored for RRM, evaluating different training paradigms and demonstrating superior performance over traditional methods.

Findings

Over 15% improvement in sum and tail rates.

CTDE balances complexity and efficiency.

Offline MARL is effective for dynamic wireless networks.

Abstract

Offline multi-agent reinforcement learning (MARL) addresses key limitations of online MARL, such as safety concerns, expensive data collection, extended training intervals, and high signaling overhead caused by online interactions with the environment. In this work, we propose an offline MARL algorithm for radio resource management (RRM), focusing on optimizing scheduling policies for multiple access points (APs) to jointly maximize the sum and tail rates of user equipment (UEs). We evaluate three training paradigms: centralized, independent, and centralized training with decentralized execution (CTDE). Our simulation results demonstrate that the proposed offline MARL framework outperforms conventional baseline approaches, achieving over a 15\% improvement in a weighted combination of sum and tail rates. Additionally, the CTDE framework strikes an effective balance, reducing the computational complexity of centralized methods while addressing the inefficiencies of independent training. These results underscore the potential of offline MARL to deliver scalable, robust, and efficient solutions for resource management in dynamic wireless networks.