Causal Inference With Survey Data: A Robust Framework for Propensity Score Weighting in Probability and Non‐Probability Samples

TL;DR

This paper introduces a new method to reduce biases in causal inference using survey data, making it more reliable for both probability and non-probability samples.

Contribution

A unified weighting framework that addresses both confounding and selection biases using survey-weighted propensity score weighting.

Findings

The proposed method provides a doubly robust inferential procedure for population weighted average treatment effects.

Key variables in external data relate to treatment effect heterogeneity and the selection mechanism.

Monte Carlo simulations and real-world applications show the method outperforms standard approaches.

Abstract

Confounding bias and selection bias are two major challenges in causal inference with observational data. While numerous methods have been developed to mitigate confounding bias, they often assume that the data are representative of the study population and ignore the potential selection bias introduced during data collection. In this paper, we propose a unified weighting framework—survey‐weighted propensity score weighting—to simultaneously address both confounding and selection biases when the observational dataset is a probability survey sample from a finite population, which is itself viewed as a random sample from the target superpopulation. The proposed method yields a doubly robust inferential procedure for a class of population weighted average treatment effects. We further extend our results to non‐probability observational data when the sampling mechanism is unknown but auxiliary information of the confounding variables is available from an external probability sample. We focus on practically important scenarios where the confounders are only partially observed in the external data. Our analysis reveals that the key variables in the external data are those related to both treatment effect heterogeneity and the selection mechanism. We also discuss how to combine auxiliary information from multiple reference probability samples. Monte Carlo simulations and an application to a real‐world non‐probability observational dataset demonstrate the superiority of our proposed methods over standard propensity score weighting approaches.