P-value: A Bless or A Curse for Evidence-Based Studies?

TL;DR

This paper explores the interpretation of p-values in hypothesis testing, establishing their equivalence with Bayesian posterior probabilities, and argues for their continued use in evidence-based research.

Contribution

It provides a Bayesian perspective on p-values, demonstrating their interpretability as posterior probabilities and advocating for their ongoing application.

Findings

P-value is equivalent to Bayesian posterior probability of the null hypothesis.

Two-sided p-values can be interpreted through two one-sided posterior probabilities.

Simulation studies support the Bayesian interpretation of p-values.

Abstract

As a convention, p-value is often computed in frequentist hypothesis testing and compared with the nominal significance level of 0.05 to determine whether or not to reject the null hypothesis. The smaller the p-value, the more significant the statistical test. We consider both one-sided and two-sided hypotheses in the composite hypothesis setting. For one-sided hypothesis tests, we establish the equivalence of p-value and the Bayesian posterior probability of the null hypothesis, which renders p-value an explicit interpretation of how strong the data support the null. For two-sided hypothesis tests of a point null, we recast the problem as a combination of two one-sided hypotheses alone the opposite directions and put forward the notion of a two-sided posterior probability, which also has an equivalent relationship with the (two-sided) p-value. Extensive simulation studies are conducted to demonstrate the Bayesian posterior probability interpretation for the p-value. Contrary to common criticisms of the use of p-value in evidence-based studies, we justify its utility and reclaim its importance from the Bayesian perspective, and recommend the continual use of p-value in hypothesis testing. After all, p-value is not all that bad.