Multi-armed Bandits for Link Configuration in Millimeter-wave Networks

TL;DR

This paper proposes using multi-armed bandit algorithms to dynamically optimize millimeter-wave link configurations, such as beam sweeping and beamwidth, improving the efficiency and adaptability of establishing mmWave links.

Contribution

It introduces a novel application of the multi-armed bandit framework to mmWave link configuration, enabling dynamic and efficient learning of optimal parameters under environmental uncertainties.

Findings

Bandit algorithms can effectively learn optimal beam parameters.

Dynamic learning improves link establishment efficiency.

Future research directions are outlined for further enhancement.

Abstract

Establishing and maintaining millimeter-wave (mmWave) links is challenging due to the changing environment and the high sensibility of mmWave signal to user mobility and channel conditions. MmWave link configuration problems often involve a search for optimal system parameter under environmental uncertainties, from a finite set of alternatives that are supported by the system hardware and protocol. For example, beam sweeping aims at identifying the optimal beam(s) for data transmission from a discrete codebook. Selecting parameters such as the beam sweeping period and the beamwidth are crucial to achieving high overall system throughput. In this article, we motivate the use of the multi-armed bandit (MAB) framework to intelligently search out the optimal configuration when establishing the mmWave links. MAB is a reinforcement learning framework that guides a decision-maker to sequentially select one action from a set of actions. As an example, we show that within the MAB framework, the optimal beam sweeping period, beamwidth, and beam directions could be dynamically learned with sample-computational-efficient bandit algorithms. We conclude by highlighting some future research directions on enhancing mmWave link configuration design with MAB.