Optimizing Adaptive Experiments: A Unified Approach to Regret Minimization and Best-Arm Identification

TL;DR

This paper introduces a unified framework for adaptive experiments that balances regret minimization during the experiment and efficient identification of the best treatment, providing theoretical insights and practical algorithms.

Contribution

It unifies regret minimization and best-arm identification in a single model, revealing how existing algorithms can be adapted for broader objectives and reducing experiment duration.

Findings

Unified model for within-experiment and post-experiment performance

Adjustment of scalar parameters in algorithms improves optimization

Significant reductions in experiment duration with minimal regret impact

Abstract

Practitioners conducting adaptive experiments often encounter two competing priorities: maximizing total welfare (or `reward') through effective treatment assignment and swiftly concluding experiments to implement population-wide treatments. Current literature addresses these priorities separately, with regret minimization studies focusing on the former and best-arm identification research on the latter. This paper bridges this divide by proposing a unified model that simultaneously accounts for within-experiment performance and post-experiment outcomes. We provide a sharp theory of optimal performance in large populations that not only unifies canonical results in the literature but also uncovers novel insights. Our theory reveals that familiar algorithms, such as the recently proposed top-two Thompson sampling algorithm, can optimize a broad class of objectives if a single scalar parameter is appropriately adjusted. In addition, we demonstrate that substantial reductions in experiment duration can often be achieved with minimal impact on both within-experiment and post-experiment regret.