Bayesian hypothesis tests with diffuse priors: Can we have our cake and eat it too?

TL;DR

This paper introduces 'cake priors' for Bayesian hypothesis testing that avoid paradoxes associated with diffuse priors, enabling valid inferences while using non-informative priors, and demonstrates their effectiveness in common statistical tests.

Contribution

The paper proposes a new class of priors called 'cake priors' that resolve Bartlett's paradox, allowing diffuse priors to be used without compromising inference validity in Bayesian hypothesis testing.

Findings

Cake priors circumvent Bartlett's paradox.

Bayesian tests with cake priors are Chernoff-consistent.

The methodology applies to t-tests and linear models.

Abstract

We introduce a new class of priors for Bayesian hypothesis testing, which we name "cake priors". These priors circumvent Bartlett's paradox (also called the Jeffreys-Lindley paradox); the problem associated with the use of diffuse priors leading to nonsensical statistical inferences. Cake priors allow the use of diffuse priors (having one's cake) while achieving theoretically justified inferences (eating it too). We demonstrate this methodology for Bayesian hypotheses tests for scenarios under which the one and two sample t-tests, and linear models are typically derived. The resulting Bayesian test statistic takes the form of a penalized likelihood ratio test statistic. By considering the sampling distribution under the null and alternative hypotheses we show for independent identically distributed regular parametric models that Bayesian hypothesis tests using cake priors are Chernoff-consistent, i.e., achieve zero type I and II errors asymptotically. Lindley's paradox is also discussed. We argue that a true Lindley's paradox will only occur with small probability for large sample sizes.