Model-Agnostic Bounds for Augmented Inverse Probability Weighted Estimators' Wald-Confidence Interval Coverage in Randomized Controlled Trials

TL;DR

This paper derives finite-sample bounds for the coverage accuracy of Wald confidence intervals for AIPW estimators in RCTs, providing guidance on variance estimation and the effects of cross-fitting.

Contribution

It introduces non-asymptotic bounds on Wald-CI coverage for AIPW estimators, analyzing bias in variance estimators and the impact of cross-fitting.

Findings

Cross-fit variance estimator may overestimate variance.

Non-cross-fit variance estimator may underestimate variance.

Cross-fitting improves Wald-CI coverage in practice.

Abstract

Nonparametric estimators, such as the augmented inverse probability weighted (AIPW) estimator, have become increasingly popular in causal inference. Numerous nonparametric estimators have been proposed, but they are all asymptotically normal with the same asymptotic variance under similar conditions, leaving little guidance for practitioners to choose an estimator. In this paper, I focus on another important perspective of their asymptotic behaviors beyond asymptotic normality, the convergence of the Wald-confidence interval (CI) coverage to the nominal coverage. Such results have been established for simpler estimators (e.g., the Berry-Esseen Theorem), but are lacking for nonparametric estimators. I consider a simple but practical setting where the AIPW estimator based on a black-box nuisance estimator, with or without cross-fitting, is used to estimate the average treatment effect in randomized controlled trials. I derive non-asymptotic Berry-Esseen-type bounds on the difference between Wald-CI coverage and the nominal coverage. I also analyze the bias of variance estimators, showing that the cross-fit variance estimator might overestimate while the non-cross-fit variance estimator might underestimate, which might explain why cross-fitting has been empirically observed to improve Wald-CI coverage.