Exploiting Side Information for Improved Online Learning Algorithms in Wireless Networks

TL;DR

This paper introduces algorithms that leverage measurable side-information in wireless networks to enhance online learning of channel throughput, significantly reducing learning time and improving throughput by exploiting correlations.

Contribution

It develops new algorithms that incorporate side-information into UCB-based online learning, quantifying the regret improvement based on correlation strength.

Findings

Exploiting side-information reduces sample complexity in learning.

Correlation between reward and side-information improves throughput.

Algorithms show significant gains in practical wireless scenarios.

Abstract

In wireless networks, the rate achieved depends on factors like level of interference, hardware impairments, and channel gain. Often, instantaneous values of some of these factors can be measured, and they provide useful information about the instantaneous rate achieved. For example, higher interference implies a lower rate. In this work, we treat any such measurable quality that has a non-zero correlation with the rate achieved as side-information and study how it can be exploited to quickly learn the channel that offers higher throughput (reward). When the mean value of the side-information is known, using control variate theory we develop algorithms that require fewer samples to learn the parameters and can improve the learning rate compared to cases where side-information is ignored. Specifically, we incorporate side-information in the classical Upper Confidence Bound (UCB) algorithm and quantify the gain achieved in the regret performance. We show that the gain is proportional to the amount of the correlation between the reward and associated side-information. We discuss in detail various side-information that can be exploited in cognitive radio and air-to-ground communication in $L-$band. We demonstrate that correlation between the reward and side-information is often strong in practice and exploiting it improves the throughput significantly.