Tight (Lower) Bounds for the Fixed Budget Best Arm Identification Bandit Problem

TL;DR

This paper establishes tight lower bounds for the fixed budget best arm identification problem in bandits, showing that existing strategies based on Successive Rejection are optimal and closing the gap between upper and lower bounds.

Contribution

It provides the first tight lower bounds for fixed budget best arm identification, disproving previous assumptions and confirming the optimality of Successive Rejection strategies.

Findings

Lower bounds show probability of error is at least exp(-T / (log(K) H))

Disproves the belief that error probability can be bounded by exp(-T/H)

Successive Rejection strategies are proven to be optimal

Abstract

We consider the problem of \textit{best arm identification} with a \textit{fixed budget $T$}, in the $K$-armed stochastic bandit setting, with arms distribution defined on $[0,1]$. We prove that any bandit strategy, for at least one bandit problem characterized by a complexity $H$, will misidentify the best arm with probability lower bounded by $$\exp\Big(-\frac{T}{\log(K)H}\Big),$$ where $H$ is the sum for all sub-optimal arms of the inverse of the squared gaps. Our result disproves formally the general belief - coming from results in the fixed confidence setting - that there must exist an algorithm for this problem whose probability of error is upper bounded by $\exp(-T/H)$. This also proves that some existing strategies based on the Successive Rejection of the arms are optimal - closing therefore the current gap between upper and lower bounds for the fixed budget best arm identification problem.