Power priors for replication studies

TL;DR

This paper introduces a Bayesian power prior method for replication studies, allowing dynamic borrowing of information from original studies based on their compatibility with new data, and explores its connection to hierarchical models.

Contribution

It proposes a novel power prior approach for Bayesian analysis in replication studies, linking it to hierarchical modeling and providing practical and theoretical insights.

Findings

Power priors effectively quantify compatibility between original and replication data.

The approach allows dynamic information borrowing based on study conflict.

Connections to hierarchical models are established, highlighting modeling nuances.

Abstract

The ongoing replication crisis in science has increased interest in the methodology of replication studies. We propose a novel Bayesian analysis approach using power priors: The likelihood of the original study's data is raised to the power of $\alpha$, and then used as the prior distribution in the analysis of the replication data. Posterior distribution and Bayes factor hypothesis tests related to the power parameter $\alpha$ quantify the degree of compatibility between the original and replication study. Inferences for other parameters, such as effect sizes, dynamically borrow information from the original study. The degree of borrowing depends on the conflict between the two studies. The practical value of the approach is illustrated on data from three replication studies, and the connection to hierarchical modeling approaches explored. We generalize the known connection between normal power priors and normal hierarchical models for fixed parameters and show that normal power prior inferences with a beta prior on the power parameter $\alpha$ align with normal hierarchical model inferences using a generalized beta prior on the relative heterogeneity variance $I^2$. The connection illustrates that power prior modeling is unnatural from the perspective of hierarchical modeling since it corresponds to specifying priors on a relative rather than an absolute heterogeneity scale.