On the relation between likelihood ratios and p-values for testing success probabilities of Bernoulli trials

TL;DR

This paper explores the asymptotic relationship between p-values and likelihood ratios in Bernoulli trials, revealing limitations on their correspondence and providing insights into their interpretation in hypothesis testing.

Contribution

It offers a detailed analysis of the asymptotic relation between p-values and likelihood ratios in coin-tossing experiments, highlighting their non-equivalence and practical bounds.

Findings

A p-value of 0.05 cannot correspond to a likelihood ratio larger than 7.5.

Large likelihood ratios are unlikely from fair coin tosses with deviations of several standard deviations.

Provides insights into the nature of p-values and likelihood ratios in Bernoulli testing.

Abstract

It is well known that there is no direct one-to-one relation between $p$-values and likelihood ratios or Bayes factors, since their relation crucially involves the sample size $n$. We investigate their (asymptotic) relation in a coin-tossing context where the hypotheses of interest address the success probability of the coin, and where detailed computations are possible. This leads to useful insights in the nature of $p$-values and likelihood ratios. Our results imply, for instance, that under mild conditions, a $p$-value of 0.05 cannot correspond to a likelihood ratio larger than 7.5, for any hypothesis versus a null hypothesis that the success probability has a specific value. We also show it is unlikely one can obtain a large likelihood ratio by tossing a fair coin until the number of heads deviates from the mean by several standard deviations.