Beamforming for Massive MIMO Aerial Communications: A Robust and Scalable DRL Approach

TL;DR

This paper introduces a robust, scalable multi-agent deep reinforcement learning framework for beamforming in massive MIMO aerial networks, improving sum-rate performance and robustness under imperfect CSI.

Contribution

It proposes a novel entropy-based multi-agent DRL approach with Fourier Neural Operators, transfer learning, and low-rank decomposition for scalable, robust beamforming in non-terrestrial networks.

Findings

Outperforms baseline schemes in sum rate and robustness.

Demonstrates scalability and efficiency in large-scale deployments.

Reduces computational and communication overhead.

Abstract

This paper presents a distributed beamforming framework for a constellation of airborne platform stations (APSs) in a massive Multiple-Input and Multiple-Output (MIMO) non-terrestrial network (NTN) that targets the downlink sum-rate maximization under imperfect local channel state information (CSI). We propose a novel entropy-based multi-agent deep reinforcement learning (DRL) approach where each non-terrestrial base station (NTBS) independently computes its beamforming vector using a Fourier Neural Operator (FNO) to capture long-range dependencies in the frequency domain. To ensure scalability and robustness, the proposed framework integrates transfer learning based on a conjugate prior mechanism and a low-rank decomposition (LRD) technique, thus enabling efficient support for large-scale user deployments and aerial layers. Our simulation results demonstrate the superiority of the proposed method over baseline schemes including WMMSE, ZF, MRT, CNN-based DRL, and the deep deterministic policy gradient (DDPG) method in terms of average sum rate, robustness to CSI imperfection, user mobility, and scalability across varying network sizes and user densities. Furthermore, we show that the proposed method achieves significant computational efficiency compared to CNN-based and WMMSE methods, while reducing communication overhead in comparison with shared-critic DRL approaches.