MAGRPO: Accelerated MARL Training for Fluid Antenna-Assisted Wireless Network Optimization

TL;DR

This paper introduces MAGRPO, a multi-agent reinforcement learning algorithm that accelerates training for optimizing fluid antenna positions, beamforming, and power in wireless networks, achieving high sum-rate performance with reduced computational complexity.

Contribution

MAGRPO replaces the critic network with group relative advantage estimation, reducing training complexity and time while maintaining performance in fluid antenna network optimization.

Findings

MAGRPO reduces training time by 30-40% compared to MAPPO.

Fluid antenna-assisted networks significantly improve sum-rate over fixed antenna systems.

MAGRPO achieves comparable sum-rates to MAPPO with lower computational cost.

Abstract

Fluid antenna system (FAS) becomes a promising paradigm for next-generation wireless networks, which enables position-flexible antenna elements that can dynamically adjust to more favorable channel conditions. However, the optimization of fluid antenna (FA) positions, beamforming, and power allocation in FA-assisted wireless networks is challenging, due to the non-convexity and the lack of base station (BS) coordination. In this paper, we first formulate this challenging optimization problem as a decentralized partially observable Markov decision process, and then propose a multi-agent group relative policy optimization (MAGRPO) algorithm under the centralized training decentralized execution (CTDE) paradigm. Compared with multi-agent proximal policy optimization (MAPPO), MAGRPO replaces the critic network with group relative advantage estimation. This design reduces computational complexity by nearly half under parameter sharing. Furthermore, we derive a variance upper bound of the cumulative reward, which scales with network parameters, e.g., the number of BSs, users, and FAs. Simulation results show that compared with wireless networks with fixed antenna positions, FA-assisted wireless networks achieve multiple-fold sum-rate enhancement. Moreover, the proposed MAGRPO attains sum-rates comparable to those of MAPPO in testing, while reducing training time by $30\% \sim 40\%$.