Fixing the Loose Brake: Exponential-Tailed Stopping Time in Best Arm Identification

TL;DR

This paper introduces new algorithms for best arm identification that ensure the stopping time has an exponential tail, improving reliability and efficiency in active experimentation.

Contribution

It proposes algorithms with provably exponential-tailed stopping times, addressing limitations of existing methods that often have heavy tails or do not stop.

Findings

Algorithms achieve exponential tail bounds on stopping time.

The first algorithm combines Sequential Halving with a doubling trick.

The second algorithm converts any fixed confidence algorithm into one with exponential tail guarantees.

Abstract

The best arm identification problem requires identifying the best alternative (i.e., arm) in active experimentation using the smallest number of experiments (i.e., arm pulls), which is crucial for cost-efficient and timely decision-making processes. In the fixed confidence setting, an algorithm must stop data-dependently and return the estimated best arm with a correctness guarantee. Since this stopping time is random, we desire its distribution to have light tails. Unfortunately, many existing studies focus on high probability or in expectation bounds on the stopping time, which allow heavy tails and, for high probability bounds, even not stopping at all. We first prove that this never-stopping event can indeed happen for some popular algorithms. Motivated by this, we propose algorithms that provably enjoy an exponential-tailed stopping time, which improves upon the polynomial tail bound reported by Kalyanakrishnan et al. (2012). The first algorithm is based on a fixed budget algorithm called Sequential Halving along with a doubling trick. The second algorithm is a meta algorithm that takes in any fixed confidence algorithm with a high probability stopping guarantee and turns it into one that enjoys an exponential-tailed stopping time. Our results imply that there is much more to be desired for contemporary fixed confidence algorithms.