A tutorial comparing different covariate balancing methods with an application evaluating the causal effects of substance use treatment programs for adolescents

TL;DR

This paper provides a comprehensive tutorial on covariate balancing methods, comparing their effectiveness through a case study on substance use treatment programs for adolescents, and offers practical guidance and tools for causal inference from observational data.

Contribution

It introduces a step-by-step framework for covariate balancing, evaluates multiple methods, and provides a user-friendly web application for practitioners.

Findings

Different covariate balancing methods vary in effectiveness depending on the context.

Proper assessment of overlap and unmeasured confounding is crucial for valid causal inference.

The case study demonstrates practical application and comparison of methods.

Abstract

Randomized controlled trials are the gold standard for measuring causal effects. However, they are often not always feasible, and causal treatment effects must be estimated from observational data. Observational studies do not allow robust conclusions about causal relationships unless statistical techniques account for the imbalance of pretreatment confounders across groups while key assumptions hold. Propensity score and balance weighting (PSBW) are useful techniques that aim to reduce the imbalances between treatment groups by weighting the groups to look alike on the observed confounders. There are many methods available to estimate PSBW. However, it is unclear a priori which will achieve the best trade-off between covariate balance and effective sample size. Moreover, it is critical to assess the validity of key assumptions required for robust estimation of the needed treatment effects, including the overlap and no unmeasured confounding assumptions. We present a step-by-step guide to covariate balancing strategies, including how to evaluate overlap, obtain estimates of PSBW, check for covariate balance, and assess sensitivity to unobserved confounding. We compare the performance of several estimation methods using a case study examining the relative effectiveness of substance use treatment programs and provide a user-friendly web application that can implement the proposed steps.