The Role of Distributional Overlap on the Precision Gain of Bounds for Generalization

TL;DR

This paper investigates how the degree of distributional overlap in propensity scores affects the ability of stratification to tighten bounds on treatment impact estimates, enhancing precision in non-randomized studies.

Contribution

It provides an empirical assessment of the impact of distributional overlap on the effectiveness of propensity score stratification in narrowing bounds.

Findings

Greater distributional overlap improves bound precision

Stratification effectiveness varies with overlap levels

Implications for causal inference in observational studies

Abstract

Over the past ten years, propensity score methods have made an important contribution to improving generalizations from studies that do not select samples randomly from a population of inference. However, these methods require assumptions and recent work has considered the role of bounding approaches that provide a range of treatment impact estimates that are consistent with the observable data. An important limitation to bound estimates is that they can be uninformatively wide. This has motivated research on the use of propensity score stratification to narrow bounds. This article assesses the role of distributional overlap in propensity scores on the effectiveness of stratification to tighten bounds. Using the results of two simulation studies and two case studies, I evaluate the relationship between distributional overlap and precision gain and discuss the implications when propensity score stratification is used as a method to improve precision in the bounding framework.