Asymptotically-Optimal Gaussian Bandits with Side Observations

TL;DR

This paper introduces an asymptotically optimal algorithm for Gaussian bandits with arbitrary side information, unifying various feedback models and establishing a fundamental regret lower bound.

Contribution

The work develops the first asymptotically optimal algorithm for Gaussian bandits with general side observations, guided by a new LP-based regret lower bound.

Findings

Established an LP-based asymptotic regret lower bound.

Designed the first asymptotically optimal algorithm for the setting.

Unified standard bandits, full-feedback, and graph feedback models.

Abstract

We study the problem of Gaussian bandits with general side information, as first introduced by Wu, Szepesvari, and Gyorgy. In this setting, the play of an arm reveals information about other arms, according to an arbitrary a priori known side information matrix: each element of this matrix encodes the fidelity of the information that the ``row'' arm reveals about the ``column'' arm. In the case of Gaussian noise, this model subsumes standard bandits, full-feedback, and graph-structured feedback as special cases. In this work, we first construct an LP-based asymptotic instance-dependent lower bound on the regret. The LP optimizes the cost (regret) required to reliably estimate the suboptimality gap of each arm. This LP lower bound motivates our main contribution: the first known asymptotically optimal algorithm for this general setting.