Choosing Covariate Balancing Methods for Causal Inference: Practical Insights from a Simulation Study

TL;DR

This study compares various covariate balancing methods for causal inference through extensive simulations, highlighting their strengths, weaknesses, and practical tuning considerations for observational data analysis.

Contribution

It provides practical insights and guidance on implementing and tuning IPTW, EB, KOM, and TLF methods based on simulation results, aiding researchers in method selection.

Findings

EB and KOM are most reliable among methods.

DR estimation reduces sensitivity to weighting schemes.

Tuning and understanding failure modes are crucial for effective use.

Abstract

Background: Inverse probability of treatment weighting (IPTW) is used for confounding adjustment in observational studies. Newer weighting methods include energy balancing (EB), kernel optimal matching (KOM), and tailored-loss covariate balancing propensity scores (TLF), but practical guidance remains limited. We evaluate their performance when implemented according to published recommendations. Methods: We conducted Monte Carlo simulations across 36 scenarios varying sample size, treatment prevalence, and a complexity factor increasing confounding and reducing overlap. Data generation used predominantly categorical covariates with some correlation. Average treatment effect and average treatment effect on the treated were estimated using IPTW, EB, KOM, and TLF combined with weighted least squares and, when supported, a doubly robust (DR) estimators. Inference followed published recommendations for each method when feasible, using standard alternatives otherwise. \textsc{PROBITsim} dataset used for illustration. Results: DR reduced sensitivity to the weighting scheme with an outcome regression adjusted for all confounders, despite functional-form misspecification. EB and KOM were most reliable; EB was tuning-free but scale dependent, whereas KOM required kernel and penalty choices. IPTW was variance sensitive when treatment prevalence was far from 50\%. TLF traded lower variance for higher bias, producing an RMSE plateau and sub-nominal confidence interval coverage. \textsc{PROBITsim} results mirrored these patterns. Conclusions: Rather than identifying a best method, our findings highlight failure modes and tuning choices to monitor. When the outcome regression adjusts for all confounders, DR estimation can be dependable across weighting schemes. Incorporating weight-estimation uncertainty into confidence intervals remains a key challenge for newer approaches.