Inference with weights: Residualization produces short, valid intervals for varying estimands and varying resampling processes

TL;DR

This paper proposes a refined method for inference in observational studies using weighting, which improves the accuracy and precision of confidence intervals by incorporating covariates and their interactions, outperforming traditional approaches.

Contribution

It introduces a residualization approach with robust standard errors that achieve valid, short confidence intervals across various resampling frameworks and weighting schemes.

Findings

Standard errors are more precise with residualization.

Method improves confidence interval accuracy under multiple frameworks.

Significant gains in precision when balancing covariates.

Abstract

Weighting procedures are used in observational causal inference to adjust for covariate imbalance within the sample. Common practice for inference is to estimate robust standard errors from a weighted regression of outcome on treatment. However, it is well known that weighting can inflate variance estimates, sometimes significantly, leading to standard errors and confidence intervals that are overly conservative. We instead examine and recommend the use of robust standard errors from a weighted regression that additionally includes the balancing covariates and their interactions with treatment. We show that these standard errors are more precise and asymptotically correct for weights that achieve exact balance under multiple common resampling frameworks, including design-based and model-based inference, as well as superpopulation sampling with a finite sample correction. Gains to precision can be quite significant when the balancing weights adjust for prognostic covariates. For procedures that balance only approximately or in expectation, such as inverse propensity weighting or approximate balancing weights, our proposed method improves precision by reducing residuals through augmentation with the parametric model. We demonstrate our approach through simulation and re-analysis of multiple empirical studies.