A response to critiques of "The reproducibility of research and the misinterpretation of p-values"

TL;DR

This paper defends the proposal to supplement p-values with false positive risk estimates, emphasizing its simplicity and interpretability, in response to critiques of prior work on research reproducibility and p-value misinterpretation.

Contribution

It clarifies and advocates for estimating false positive risk alongside p-values, addressing critiques and emphasizing the method's simplicity and interpretability.

Findings

FPR provides a Bayesian measure of mistaken claims based on p-values.

The proposed FPR estimation method is simpler and more understandable.

The paper responds to critiques, reinforcing the value of FPR in research interpretation.

Abstract

I proposed (8, 1, 3) that p values should be supplemented by an estimate of the false positive risk (FPR). FPR was defined as the probability that, if you claim that there is a real effect on the basis of p value from a single unbiased experiment, that you will be mistaken and the result has occurred by chance. This is a Bayesian quantity and that means that there is an infinitude of ways to calculate it. My choice of a way to estimate FPR was, therefore, arbitrary. I maintain that it is a reasonable way, and has the advantage of being mathematically simpler than other proposals and easier to understand than other methods. This might make it more easily accepted by users. As always, not every statistician agrees. This paper is a response to a critique of my 2017 paper (1) by Arandjelovic (2)