Continuous time asymptotic representations for adaptive experiments

TL;DR

This paper introduces a continuous-time asymptotic framework for adaptive experiments, enabling better analysis of treatment allocation and inference in dynamically evolving data collection settings.

Contribution

It develops a Gaussian diffusion limit for empirical allocation processes in adaptive experiments, simplifying analysis and enabling new valid inference methods.

Findings

Gaussian diffusion approximation for adaptive allocation

Framework for optimal estimators and regret analysis

First valid anytime inference for multi-treatment experiments

Abstract

This article develops a continuous-time asymptotic framework for analyzing adaptive experiments -- settings in which data collection and treatment assignment evolve dynamically in response to incoming information. A key challenge in analyzing fully adaptive experiments, where the assignment policy is updated after each observation, is that the sequence of policy rules often lack a well-defined asymptotic limit. To address this, we focus instead on the empirical allocation process, which captures the fraction of observations assigned to each treatment over time. We show that, under general conditions, any adaptive experiment and its associated empirical allocation process can be approximated by a limit experiment defined by Gaussian diffusions with unknown drifts and a corresponding continuous-time allocation process. This limit representation facilitates the analysis of optimal decision rules by reducing the dimensionality of the state-space and leveraging the tractability of Gaussian diffusions. We apply the framework to derive optimal estimators, analyze in-sample regret for adaptive experiments, and construct e-processes for anytime-valid inference. Notably, we introduce the first definition of any-time and any-experiment valid inference for multi-treatment settings.