Multi-Agent Double Deep Q-Learning for Beamforming in mmWave MIMO Networks

TL;DR

This paper introduces a distributed multi-agent double deep Q-learning approach for beamforming in mmWave MIMO networks, enabling base stations to dynamically optimize beams for highly-mobile users, achieving high performance with reduced complexity.

Contribution

The paper presents a novel multi-agent deep reinforcement learning algorithm for beamforming in mmWave MIMO networks, addressing mobility and complexity challenges.

Findings

Achieves comparable performance to exhaustive search

Operates at significantly lower complexity

Effective in highly-mobile user scenarios

Abstract

Beamforming is one of the key techniques in millimeter wave (mmWave) multi-input multi-output (MIMO) communications. Designing appropriate beamforming not only improves the quality and strength of the received signal, but also can help reduce the interference, consequently enhancing the data rate. In this paper, we propose a distributed multi-agent double deep Q-learning algorithm for beamforming in mmWave MIMO networks, where multiple base stations (BSs) can automatically and dynamically adjust their beams to serve multiple highly-mobile user equipments (UEs). In the analysis, largest received power association criterion is considered for UEs, and a realistic channel model is taken into account. Simulation results demonstrate that the proposed learning-based algorithm can achieve comparable performance with respect to exhaustive search while operating at much lower complexity.