Balancing Performance and Costs in Best Arm Identification

TL;DR

This paper introduces a new framework for best arm identification in multi-armed bandits that balances performance and costs, providing theoretical bounds and an algorithm that outperforms traditional methods in practical scenarios.

Contribution

It proposes a novel risk-based formalism for best arm identification that explicitly balances performance and costs, along with a matching algorithm and theoretical bounds.

Findings

Theoretical lower bounds for risk in new formalism

DBCARE algorithm matches bounds up to polylog factors

Demonstrates improved performance over classical methods in simulations

Abstract

We consider the problem of identifying the best arm in a multi-armed bandit model. Despite a wealth of literature in the traditional fixed budget and fixed confidence regimes of the best arm identification problem, it still remains a mystery to most practitioners as to how to choose an approach and corresponding budget or confidence parameter. We propose a new formalism to avoid this dilemma altogether by minimizing a risk functional which explicitly balances the performance of the recommended arm and the cost incurred by learning this arm. In this framework, a cost is incurred for each observation during the sampling phase, and upon recommending an arm, a performance penalty is incurred for identifying a suboptimal arm. The learner's goal is to minimize the sum of the penalty and cost. This new regime mirrors the priorities of many practitioners, e.g. maximizing profit in an A/B testing framework, better than classical fixed budget or confidence settings. We derive theoretical lower bounds for the risk of each of two choices for the performance penalty, the probability of misidentification and the simple regret, and propose an algorithm called DBCARE to match these lower bounds up to polylog factors on nearly all problem instances. We then demonstrate the performance of DBCARE on a number of simulated models, comparing to fixed budget and confidence algorithms to show the shortfalls of existing BAI paradigms on this problem.