Budgeted Active Experimentation for Treatment Effect Estimation from Observational and Randomized Data

TL;DR

This paper introduces a budgeted active experimentation method that combines observational data and targeted randomized trials to improve treatment effect estimation efficiently under budget constraints.

Contribution

It develops an active sampling framework leveraging observational priors and theoretical bounds to optimize experimental design for causal inference.

Findings

Outperforms standard randomized baselines in cost-constrained scenarios

Provides finite-sample deviation bounds and asymptotic normality proofs

Achieves minimax optimality in information-theoretic sense

Abstract

Estimating heterogeneous treatment effects is central to data-driven decision-making, yet industrial applications often face a fundamental tension between limited randomized controlled trial (RCT) budgets and abundant but biased observational data collected under historical targeting policies. Although observational logs offer the advantage of scale, they inherently suffer from severe policyinduced imbalance and overlap violations, rendering standalone estimation unreliable. We propose a budgeted active experimentation framework that iteratively enhances model training for causal effect estimation via active sampling. By leveraging observational priors, we develop an acquisition function targeting uplift estimation uncertainty, overlap deficits, and domain discrepancy to select the most informative units for randomized experiments. We establish finite-sample deviation bounds, asymptotic normality via martingale Central Limit Theorems (CLTs), and minimax lower bounds to prove information-theoretic optimality. Extensive experiments on industrial datasets demonstrate that our approach significantly outperforms standard randomized baselines in cost-constrained settings.