Demystify Doubly-Robust Estimation: The Role of Overlap

TL;DR

This paper investigates how the finite sample performance of the doubly-robust estimator in causal inference is affected by covariate overlap, revealing limitations under poor overlap and providing practical guidance for analysts.

Contribution

The study provides a detailed analysis of the finite sample behavior of the doubly-robust estimator, highlighting the impact of covariate overlap and offering practical recommendations.

Findings

Outcome model specification influences DR estimates more than propensity score model.

Poor overlap amplifies bias and variance in DR estimates, often making them worse than IPW or outcome models.

Checking and addressing covariate overlap is crucial for reliable causal inference.

Abstract

The doubly-robust (DR) estimator is popular for evaluating causal effects in observational studies and is often perceived as more desirable than inverse probability weighting (IPW) or outcome modeling alone because it provides extra protection against model misspecification. However, double robustness is an asymptotic property that may not hold in finite samples. We investigate how the finite sample performance of the DR estimator depends on the degree of covariate overlap between comparison groups. Using analytical illustrations and extensive simulations under various scenarios with different degrees of covariate overlap and model specifications, we examine the bias and variance of the DR estimator relative to IPW and outcome modeling estimators. We find that: (i) specification of the outcome model has a stronger influence on the DR estimates than specification of the propensity score model, and this dominance increases as overlap decreases; (ii) with poor overlap, the DR estimator generally amplifies the adverse consequences of extreme weights (large bias and/or variance) regardless of model specifications, and is often inferior to both the IPW and outcome modeling estimators. As a practical guide, we recommend always first checking the degree of overlap in applications. In the case of poor overlap, analysts should consider shifting the target population to a subpopulation with adequate overlap via methods such as trimming or overlap weighting.