Is the p-value a good measure of evidence? An asymptotic consistency criterion

TL;DR

This paper evaluates the effectiveness of p-values as a measure of statistical evidence, proposing that consistency is a key criterion, and finds that p-values are not consistent while likelihood ratios are.

Contribution

It introduces the concept of consistency as a criterion for evidence measures and demonstrates that p-values lack this property, unlike likelihood ratios.

Findings

P-values are not consistent as evidence measures.

Likelihood ratios are consistent and reliable.

Consistency should be a key criterion for evaluating evidence measures.

Abstract

What are the criteria that a measure of statistical evidence should satisfy? It is argued that a measure of evidence should be consistent. Consistency is an asymptotic criterion: the probability that if a measure of evidence in data strongly testifies against a hypothesis H, then H is indeed not true, should go to one, as more and more data appear. The p-value is not consistent, while the ratio of likelihoods is.