Bridging the gap between regret minimization and best arm identification, with application to A/B tests

TL;DR

This paper introduces algorithms that simultaneously minimize regret and identify the best option quickly, bridging a gap between two common objectives in online learning, with applications in A/B testing and clinical trials.

Contribution

It provides a theoretical framework for algorithms that achieve both regret minimization and delta-PAC guarantees, extending to non-iid data scenarios.

Findings

Algorithms achieve guaranteed decision times while minimizing regret.

Ill-callibrated UCB algorithms are effective in quick best arm identification.

The approach applies to adaptive testing in various practical domains.

Abstract

State of the art online learning procedures focus either on selecting the best alternative ("best arm identification") or on minimizing the cost (the "regret"). We merge these two objectives by providing the theoretical analysis of cost minimizing algorithms that are also delta-PAC (with a proven guaranteed bound on the decision time), hence fulfilling at the same time regret minimization and best arm identification. This analysis sheds light on the common observation that ill-callibrated UCB-algorithms minimize regret while still identifying quickly the best arm. We also extend these results to the non-iid case faced by many practitioners. This provides a technique to make cost versus decision time compromise when doing adaptive tests with applications ranging from website A/B testing to clinical trials.