An Empirical Process Approach to the Union Bound: Practical Algorithms for Combinatorial and Linear Bandits

TL;DR

This paper introduces near-optimal algorithms for linear bandit pure-exploration problems, utilizing empirical process theory to improve sample complexity bounds and extend to fixed budget settings, especially for combinatorial classes.

Contribution

It develops a novel experimental design based on Gaussian-width, providing tight bounds and efficient algorithms for combinatorial linear bandits in both fixed confidence and fixed budget scenarios.

Findings

Sample complexity scales with instance geometry

Algorithm matches lower bounds in fixed confidence setting

First fixed budget linear bandit algorithm with near-optimal guarantees

Abstract

This paper proposes near-optimal algorithms for the pure-exploration linear bandit problem in the fixed confidence and fixed budget settings. Leveraging ideas from the theory of suprema of empirical processes, we provide an algorithm whose sample complexity scales with the geometry of the instance and avoids an explicit union bound over the number of arms. Unlike previous approaches which sample based on minimizing a worst-case variance (e.g. G-optimal design), we define an experimental design objective based on the Gaussian-width of the underlying arm set. We provide a novel lower bound in terms of this objective that highlights its fundamental role in the sample complexity. The sample complexity of our fixed confidence algorithm matches this lower bound, and in addition is computationally efficient for combinatorial classes, e.g. shortest-path, matchings and matroids, where the arm sets can be exponentially large in the dimension. Finally, we propose the first algorithm for linear bandits in the the fixed budget setting. Its guarantee matches our lower bound up to logarithmic factors.