Online Balanced Experimental Design

TL;DR

This paper introduces online algorithms for experimental design that minimize covariate imbalance and variance, improving precision and robustness in randomized experiments with multiple treatments.

Contribution

It presents computationally efficient online algorithms that handle arbitrary treatment probabilities, incorporate randomization, and provide theoretical bounds on estimation error.

Findings

Algorithms outperform complete randomization in simulations

Proposed methods achieve lower covariate imbalance

Theoretical bounds are within a logarithmic factor of offline optimal

Abstract

e consider the experimental design problem in an online environment, an important practical task for reducing the variance of estimates in randomized experiments which allows for greater precision, and in turn, improved decision making. In this work, we present algorithms that build on recent advances in online discrepancy minimization which accommodate both arbitrary treatment probabilities and multiple treatments. The proposed algorithms are computational efficient, minimize covariate imbalance, and include randomization which enables robustness to misspecification. We provide worst case bounds on the expected mean squared error of the causal estimate and show that the proposed estimator is no worse than an implicit ridge regression, which are within a logarithmic factor of the best known results for offline experimental design. We conclude with a detailed simulation study showing favorable results relative to complete randomization as well as to offline methods for experimental design with time complexities exceeding our algorithm.