Optimal Best Arm Identification in Two-Armed Bandits with a Fixed Budget under a Small Gap

TL;DR

This paper demonstrates that a Neyman allocation-based strategy is asymptotically optimal for fixed-budget best-arm identification in two-armed Gaussian bandits with small reward gaps, matching theoretical lower bounds.

Contribution

It introduces the NA-AIPW strategy combining Neyman allocation with AIPW estimation, proving its asymptotic optimality in small-gap scenarios.

Findings

The proposed strategy achieves matching upper and lower bounds asymptotically.

Neyman allocation effectively balances exploration and exploitation in small-gap settings.

The strategy is proven optimal as the budget increases and the gap decreases.

Abstract

We consider fixed-budget best-arm identification in two-armed Gaussian bandit problems. One of the longstanding open questions is the existence of an optimal strategy under which the probability of misidentification matches a lower bound. We show that a strategy following the Neyman allocation rule (Neyman, 1934) is asymptotically optimal when the gap between the expected rewards is small. First, we review a lower bound derived by Kaufmann et al. (2016). Then, we propose the "Neyman Allocation (NA)-Augmented Inverse Probability weighting (AIPW)" strategy, which consists of the sampling rule using the Neyman allocation with an estimated standard deviation and the recommendation rule using an AIPW estimator. Our proposed strategy is optimal because the upper bound matches the lower bound when the budget goes to infinity and the gap goes to zero.