The Letter Pi : Bayesian interpretation of p-values, Reproducibility and Considerations for Replication in the Generalized Linear Model

TL;DR

This paper proposes a Bayesian interpretation of p-values called pi-values for Generalized Linear Models, linking Bayesian and frequentist approaches to improve reproducibility and decision-making in scientific research.

Contribution

It introduces a decision framework for thresholding posterior tail areas, connecting p-values with Bayesian posterior probabilities in GLMs, and offers a synthesis of likelihood and Bayesian methods.

Findings

Pi-values are non-controversial Bayesian summaries of effects.

The framework links p-values with posterior tail probabilities.

Application to clinical trial data demonstrates practical utility.

Abstract

Significance testing based on p-values has been implicated in the reproducibility crisis in scientific research, with one of the proposals being to eliminate them in favor of Bayesian analyses. Defenders of the p-values have countered that it is the improper use and errors in interpretation, rather than the p-values themselves that are to blame. Similar exchanges about the role of p-values have occurred with some regularity every 10 to 15 years since their formal introduction in statistical practice. The apparent contradiction between the repeated failures in interpretation and continuous use of p-values suggest that there is an inferential value in the computation of these values. In this work we propose to attach a radical Bayesian interpretation to the number computed and reported as a p-value for the Generalized Linear Model, which has been the workhorse of applied statistical work. We introduce a decision analytic framework for thresholding posterior tail areas (pi-values) which for any given Bayesian analysis will have a direct correspondence to p-values in non-Bayesian approaches. Pi-values are non-controversial, posterior probability summaries of treatment effects. A predictive probability argument is made to justify the exploration of the stochastic variation (replication probability) of p and pi-values and culminates into a concrete proposal for the synthesis of Likelihood and Bayesian approaches to data analyses that aim for reproducibility. We illustrate these concepts using the results of recent randomized controlled trials in cardiometabolic and kidney diseases and provide R code for the implementation of the proposed methodology.