A unified analysis of regression adjustment in randomized experiments

TL;DR

This paper provides a unified theoretical framework to compare the asymptotic efficiency of various regression-adjusted estimators in randomized experiments, revealing conditions under which adjustments improve precision.

Contribution

It introduces a comprehensive analysis of the asymptotic variance of linear regression-adjusted estimators without assuming correct model specification, extending to various treatment mechanisms.

Findings

Certain regression adjustments guarantee increased efficiency under specific conditions.

Variance dominance among estimators depends on treatment assignment and model assumptions.

The variance advantage may not hold in generalized linear models or complex treatment schemes.

Abstract

Regression adjustment is broadly applied in randomized trials under the premise that it usually improves the precision of a treatment effect estimator. However, previous work has shown that this is not always true. To further understand this phenomenon, we develop a unified comparison of the asymptotic variance of a class of linear regression-adjusted estimators. Our analysis is based on the classical theory for linear regression with heteroscedastic errors and thus does not assume that the postulated linear model is correct. For a completely randomized binary treatment, we provide sufficient conditions under which some regression-adjusted estimators are guaranteed to be more asymptotically efficient than others. We explore other settings such as general treatment assignment mechanisms and generalized linear models, and find that the variance dominance phenomenon no longer occurs.