Mobile Cell-Free Massive MIMO with Multi-Agent Reinforcement Learning: A Scalable Framework

TL;DR

This paper introduces a scalable multi-agent reinforcement learning framework with graph neural networks for mobile cell-free massive MIMO systems, significantly improving coverage, interference management, and system stability in dynamic environments.

Contribution

It proposes a novel MARL-based framework with GNNs and advanced architectures to enhance scalability, collaboration, and performance in mobile cell-free mMIMO systems.

Findings

Enhanced system performance over conventional schemes

Effective collaboration via GNN-aided communication

Balanced performance and convergence through observation compression

Abstract

Cell-free massive multiple-input multiple-output (mMIMO) offers significant advantages in mobility scenarios, mainly due to the elimination of cell boundaries and strong macro diversity. In this paper, we examine the downlink performance of cell-free mMIMO systems equipped with mobile-APs utilizing the concept of unmanned aerial vehicles, where mobility and power control are jointly considered to effectively enhance coverage and suppress interference. However, the high computational complexity, poor collaboration, limited scalability, and uneven reward distribution of conventional optimization schemes lead to serious performance degradation and instability. These factors complicate the provision of consistent and high-quality service across all user equipments in downlink cell-free mMIMO systems. Consequently, we propose a novel scalable framework enhanced by multi-agent reinforcement learning (MARL) to tackle these challenges. The established framework incorporates a graph neural network (GNN)-aided communication mechanism to facilitate effective collaboration among agents, a permutation architecture to improve scalability, and a directional decoupling architecture to accurately distinguish contributions. In the numerical results, we present comparisons of different optimization schemes and network architectures, which reveal that the proposed scheme can effectively enhance system performance compared to conventional schemes due to the adoption of advanced technologies. In particular, appropriately compressing the observation space of agents is beneficial for achieving a better balance between performance and convergence.