Rate-optimal Design for Anytime Best Arm Identification

TL;DR

This paper introduces a new, practical algorithm called Almost Tracking for best arm identification that is minimax optimal, flexible, and outperforms existing methods in various settings.

Contribution

It proposes a novel, closed-form algorithm for anytime best arm identification that is minimax optimal and does not require pre-allocating the total budget.

Findings

Almost Tracking outperforms existing algorithms in experiments.

The algorithm is minimax optimal up to a constant factor.

It does not require pre-allocating the total sampling budget.

Abstract

We consider the best arm identification problem, where the goal is to identify the arm with the highest mean reward from a set of $K$ arms under a limited sampling budget. This problem models many practical scenarios such as A/B testing. We consider a class of algorithms for this problem, which is provably minimax optimal up to a constant factor. This idea is a generalization of existing works in fixed-budget best arm identification, which are limited to a particular choice of risk measures. Based on the framework, we propose Almost Tracking, a closed-form algorithm that has a provable guarantee on the popular risk measure $H_1$. Unlike existing algorithms, Almost Tracking does not require the total budget in advance nor does it need to discard a significant part of samples, which gives a practical advantage. Through experiments on synthetic and real-world datasets, we show that our algorithm outperforms existing anytime algorithms as well as fixed-budget algorithms.