Posterior Predictive Propensity Scores and $p$-Values

TL;DR

This paper introduces a Bayesian approach to causal inference using posterior predictive p-values that incorporate propensity scores, reconciling Bayesian and frequentist perspectives and improving finite-sample performance.

Contribution

It proposes a novel Bayesian method utilizing posterior predictive p-values with propensity scores, addressing previous incoherence and enhancing causal effect testing.

Findings

The proposed p-value equals the Fisher randomization test p-value averaged over the posterior.

Using a studentized doubly robust estimator, the p-value inherits robustness and asymptotic validity.

The method can outperform traditional frequentist p-values in finite samples, especially with extreme propensity scores.

Abstract

\citet{Rosenbaum83ps} introduced the notion of the propensity score and discussed its central role in causal inference with observational studies. Their paper, however, caused a fundamental incoherence with an early paper by \citet{Rubin78}, which showed that the propensity score does not play any role in the Bayesian analysis of unconfounded observational studies if the priors on the propensity score and outcome models are independent. Despite the serious efforts made in the literature, it is generally difficult to reconcile these contradicting results. We offer a simple approach to incorporating the propensity score in Bayesian causal inference based on the posterior predictive $p$-value. To motivate a simple procedure, we focus on the model with the strong null hypothesis of no causal effects for any units whatsoever. Computationally, the proposed posterior predictive $p$-value equals the classic $p$-value based on the Fisher randomization test averaged over the posterior predictive distribution of the propensity score. Moreover, using the studentized doubly robust estimator as the test statistic, the proposed $p$-value inherits the doubly robust property and is also asymptotically valid for testing the weak null hypothesis of zero average causal effect. Perhaps surprisingly, this Bayesianly motivated $p$-value can have better frequentist's finite-sample performance than the frequentist's $p$-value based on the asymptotic approximation especially when the propensity scores can take extreme values.