Non-Asymptotic Analysis of (Sticky) Track-and-Stop

TL;DR

This paper provides non-asymptotic performance guarantees for the Track-and-Stop and Sticky Track-and-Stop algorithms in pure exploration problems, extending their theoretical understanding beyond asymptotic regimes.

Contribution

It offers the first non-asymptotic analysis of these algorithms, including the Sticky variant, in environments with multiple correct answers.

Findings

Non-asymptotic guarantees established for Track-and-Stop.

Non-asymptotic guarantees established for Sticky Track-and-Stop.

Results extend understanding of algorithm performance in finite-sample settings.

Abstract

In pure exploration problems, a statistician sequentially collects information to answer a question about some stochastic and unknown environment. The probability of returning a wrong answer should not exceed a maximum risk parameter $\delta$ and good algorithms make as few queries to the environment as possible. The Track-and-Stop algorithm is a pioneering method to solve these problems. Specifically, it is well-known that it enjoys asymptotic optimality sample complexity guarantees for $\delta\to 0$ whenever the map from the environment to its correct answers is single-valued (e.g., best-arm identification with a unique optimal arm). The Sticky Track-and-Stop algorithm extends these results to settings where, for each environment, there might exist multiple correct answers (e.g., $\epsilon$-optimal arm identification). Although both methods are optimal in the asymptotic regime, their non-asymptotic guarantees remain unknown. In this work, we fill this gap and provide non-asymptotic guarantees for both algorithms.