Unbiased Regression-Adjusted Estimation of Average Treatment Effects in Randomized Controlled Trials

TL;DR

This paper proposes LOORA, a leave-one-out regression adjustment method for unbiased estimation of average treatment effects in randomized trials, improving bias and variance properties especially in small samples.

Contribution

Introduces LOORA, a novel leave-one-out regression adjustment technique that reduces bias and enhances stability in treatment effect estimation.

Findings

LOORA removes substantial bias in finite samples.

LOORA provides exact variance formulas for adjusted estimators.

LOORA achieves near-nominal confidence interval coverage in real experiments.

Abstract

This article introduces a leave-one-out regression adjustment (LOORA) for estimating average treatment effects in randomized controlled trials. In finite samples, LOORA removes the bias of conventional regression adjustment and yields exact variance formulas for regression-adjusted Horvitz-Thompson and difference-in-means estimators. Ridge regularization curbs the influence of high-leverage observations, improving stability and precision in small samples. In large samples, LOORA matches the variance of the regression-adjusted estimator in Lin (2013) while remaining exactly unbiased. Two within-subject experimental applications, each providing a realistic joint distribution of potential outcomes as ground truth, show that LOORA removes substantial bias and achieves confidence interval coverage close to the nominal level.