Reinforcement Learning-based Joint Handover and Beam Tracking in Millimeter-wave Networks

TL;DR

This paper introduces a reinforcement learning algorithm for joint handover and beam tracking in mmWave networks, improving connection reliability and reducing handovers by intelligently deciding when and how to switch beams and base stations.

Contribution

The paper presents a novel RL-based method for joint handover and beam tracking, optimizing connection stability and throughput in mmWave networks.

Findings

Outperforms baseline methods in achievable throughput.

Reduces the number of handovers needed.

Effective in outdoor environments.

Abstract

In this paper, we develop an algorithm for joint handover and beam tracking in millimeter-wave (mmWave) networks. The aim is to provide a reliable connection in terms of the achieved throughput along the trajectory of the mobile user while preventing frequent handovers. We model the association problem as an optimization problem and propose a reinforcement learning-based solution. Our approach learns whether and when beam tracking and handover should be performed and chooses the target base stations. In the case of beam tracking, we propose a tracking algorithm based on measuring a small spatial neighbourhood of the optimal beams in the previous time slot. Simulation results in an outdoor environment show the superior performance of our proposed solution in achievable throughput and the number of handovers needed in comparison to a multi-connectivity baseline and a learning-based handover baseline.