A Non-asymptotic Approach to Best-Arm Identification for Gaussian Bandits

TL;DR

This paper introduces Exploration-Biased Sampling, a new non-asymptotic strategy for Gaussian bandit best-arm identification that improves exploration stability and matches asymptotic optimality.

Contribution

It presents the first non-asymptotic bounds for a strategy that asymptotically matches sample complexity, with enhanced exploration behavior and a novel analysis approach.

Findings

Strategy is asymptotically optimal

Improved exploration stability and interpretability

Faster numerical resolution of sample complexity optimization

Abstract

We propose a new strategy for best-arm identification with fixed confidence of Gaussian variables with bounded means and unit variance. This strategy, called Exploration-Biased Sampling, is not only asymptotically optimal: it is to the best of our knowledge the first strategy with non-asymptotic bounds that asymptotically matches the sample complexity.But the main advantage over other algorithms like Track-and-Stop is an improved behavior regarding exploration: Exploration-Biased Sampling is biased towards exploration in a subtle but natural way that makes it more stable and interpretable. These improvements are allowed by a new analysis of the sample complexity optimization problem, which yields a faster numerical resolution scheme and several quantitative regularity results that we believe of high independent interest.