Simulation-Based Inference for Adaptive Experiments

TL;DR

This paper introduces a simulation-based inference method for adaptive multi-arm bandit experiments, providing more accurate confidence intervals and hypothesis tests than existing approaches.

Contribution

We propose a novel simulation with optimism technique for inference in adaptive experiments, overcoming limitations of asymptotic and martingale-based methods.

Findings

Achieves desired coverage in confidence intervals

Reduces confidence interval widths by up to 50%

Improves inference accuracy for non-targeted arms

Abstract

Multi-arm bandit experimental designs are increasingly being adopted over standard randomized trials due to their potential to improve outcomes for study participants, enable faster identification of the best-performing options, and/or enhance the precision of estimating key parameters. Current approaches for inference after adaptive sampling either rely on asymptotic normality under restricted experiment designs or underpowered martingale concentration inequalities that lead to weak power in practice. To bypass these limitations, we propose a simulation-based approach for conducting hypothesis tests and constructing confidence intervals for arm specific means and their differences. Our simulation-based approach uses positively biased nuisances to generate additional trajectories of the experiment, which we call \textit{simulation with optimism}. Using these simulations, we characterize the distribution potentially non-normal sample mean test statistic to conduct inference. We provide guarantees for (i) asymptotic type I error control, (ii) convergence of our confidence intervals, and (iii) asymptotic strong consistency of our estimator over a wide variety of common bandit designs. Our empirical results show that our approach achieves the desired coverage while reducing confidence interval widths by up to 50%, with drastic improvements for arms not targeted by the design.