Multi-agent Reinforcement Learning with Graph Q-Networks for Antenna Tuning

TL;DR

This paper introduces a multi-agent reinforcement learning method using graph neural networks to optimize antenna configurations in large mobile networks, demonstrating improved global performance in simulations.

Contribution

It presents a novel multi-agent RL algorithm with graph neural networks that generalizes across network topologies and learns coordination for antenna tuning.

Findings

Effective in antenna tilt tuning in simulations

Outperforms local optimization methods

Generalizes to different network topologies

Abstract

Future generations of mobile networks are expected to contain more and more antennas with growing complexity and more parameters. Optimizing these parameters is necessary for ensuring the good performance of the network. The scale of mobile networks makes it challenging to optimize antenna parameters using manual intervention or hand-engineered strategies. Reinforcement learning is a promising technique to address this challenge but existing methods often use local optimizations to scale to large network deployments. We propose a new multi-agent reinforcement learning algorithm to optimize mobile network configurations globally. By using a value decomposition approach, our algorithm can be trained from a global reward function instead of relying on an ad-hoc decomposition of the network performance across the different cells. The algorithm uses a graph neural network architecture which generalizes to different network topologies and learns coordination behaviors. We empirically demonstrate the performance of the algorithm on an antenna tilt tuning problem and a joint tilt and power control problem in a simulated environment.