Online Learning for Position-Aided Millimeter Wave Beam Training

TL;DR

This paper introduces online learning algorithms based on multi-armed bandit frameworks to efficiently perform beam training in millimeter wave communications, significantly reducing overhead and improving beam alignment accuracy.

Contribution

It develops novel online learning algorithms for beam pair selection and refinement using a multi-armed bandit approach with risk-aware features and optimistic optimization.

Findings

Achieves 1dB gain over exhaustive search within 100 steps

Uses only 30 beam pairs for training, reducing overhead

Effective for 16x16 antenna arrays in millimeter wave systems

Abstract

Accurate beam alignment is essential for beam-based millimeter wave communications. Conventional beam sweeping solutions often have large overhead, which is unacceptable for mobile applications like vehicle-to-everything. Learning-based solutions that leverage sensor data like position to identify good beam directions are one approach to reduce the overhead. Most existing solutions, though, are supervised-learning where the training data is collected beforehand. In this paper, we use a multi-armed bandit framework to develop online learning algorithms for beam pair selection and refinement. The beam pair selection algorithm learns coarse beam directions in some predefined beam codebook, e.g., in discrete angles separated by the 3dB beamwidths. The beam refinement fine-tunes the identified directions to match the peak of the power angular spectrum at that position. The beam pair selection uses the upper confidence bound (UCB) with a newly proposed risk-aware feature, while the beam refinement uses a modified optimistic optimization algorithm. The proposed algorithms learn to recommend good beam pairs quickly. When using 16x16 arrays at both the transmitter and receiver, it can achieve on average 1dB gain over the exhaustive search (over 271x271 beam pairs) on the unrefined codebook within 100 time-steps with a training budget of only 30 beam pairs.