Propensity Score Methods for Merging Observational and Experimental Datasets

TL;DR

This paper develops methods to combine limited randomized trial data with larger observational data using propensity scores, improving causal effect estimation by leveraging the strengths of both data sources.

Contribution

It introduces two novel methods for merging RCT and observational data based on propensity score stratification, enhancing robustness and external validity of causal estimates.

Findings

The spike-in method performs best when RCT and ODB covariates are similarly distributed.

The convex combination method is more robust to covariate bias.

Application to Women's Health Initiative data shows stable causal estimates of hormone therapy effects.

Abstract

This project considers how one might augment a limited amount of data from randomized controlled trial (RCT) with more plentiful data from an observational database (ODB), in order to estimate a causal effect. In our motivating setting, the ODB has better external validity, while the RCT has genuine randomization. We work with strata defined by the propensity score in the ODB. Subjects from the RCT are placed in strata defined by the propensity they would have had, had they been in the ODB. Our first method simply spikes the RCT data into their corresponding ODB strata. Our second method takes a data-driven convex combination of the ODB and RCT treatment effect estimates within each stratum. Using the delta method and simulations we show that the spike-in method works best when the RCT covariates are drawn from the same distribution as in the ODB. Our convex combination method is more robust than the spike-in to covariate-based inclusion criteria that bias the RCT data. We apply our methods to data from the Women's Health Initiative, a study of thousands of postmenopausal women which has both observational and experimental data on hormone therapy (HT). Using half of the RCT to define a gold standard, we find that a version of the spiked-in estimate yields stable estimates of the causal impact of HT on coronary heart disease.