Confounding Adjustment Methods for Multi-level Treatment Comparisons Under Lack of Positivity and Unknown Model Specification

TL;DR

This paper evaluates various confounding adjustment methods for multi-level treatments in observational studies, especially under challenges like positivity violations and unknown models, proposing a new variance estimator and demonstrating its effectiveness through simulations and real data analysis.

Contribution

It identifies the most robust adjustment methods under challenging conditions and introduces a simple variance estimator compatible with machine learning, enhancing bias correction in complex scenarios.

Findings

Overlap weights and augmented overlap weights perform well under violations.

Outcome modeling improves adjustment accuracy over treatment-only models.

Machine learning enhances bias correction for outcome-including methods.

Abstract

Imbalances in covariates between treatment groups are frequent in observational studies and can lead to biased comparisons. Various adjustment methods can be employed to correct these biases in the context of multi-level treatments ($>$ 2). However, analytical challenges, such as positivity violations and incorrect model specification, may affect their ability to yield unbiased estimates. Adjustment methods that present the best potential to deal with those challenges were identified: the overlap weights, augmented overlap weights, bias-corrected matching and targeted maximum likelihood. A simple variance estimator for the overlap weight estimators that can naturally be combined with machine learning algorithms is proposed. In a simulation study, we investigated the empirical performance of these methods as well as those of simpler alternatives, standardization, inverse probability weighting and matching. Our proposed variance estimator performed well, even at a sample size of 500. Adjustment methods that included an outcome modeling component performed better than those that only modeled the treatment mechanism. Additionally, a machine learning implementation was observed to efficiently compensate for the unknown model specification for the former methods, but not the latter. Based on these results, the wildfire data were analyzed using the augmented overlap weight estimator. With respect to effectiveness of alternate fire-suppression interventions, the results were counter-intuitive, indeed the opposite of what would be expected on subject-matter grounds. This suggests the presence in the data of unmeasured confounding bias.