On the Equivalence between Bayesian and Classical Hypothesis Testing

TL;DR

This paper demonstrates the equivalence between Bayesian and classical hypothesis tests for simple hypotheses, showing that the Bayes factor test's power matches the classical test once the type I error is fixed, and explores extensions to complex models.

Contribution

It establishes the conditions under which Bayesian and classical tests are equivalent and analyzes the impact of priors on test power in complex models.

Findings

Bayes factor tests have the same power as classical tests when type I error is fixed.

Using Jeffreys prior on nuisance parameters recovers classical test behavior.

Proper priors on nuisance parameters lead to tests with lower power than classical tests.

Abstract

For hypotheses of the type H_0:theta=theta_0 vs H_1:theta ne theta_0 we demonstrate the equivalence of a Bayesian hypothesis test using a Bayes factor and the corresponding classical test, for a large class of models, which are detailed in the paper. In particular, we show that the role of the prior and critical region for the Bayes factor test is only to specify the type I error. This is their only role since, as we show, the power function of the Bayes factor test coincides exactly with that of the classical test, once the type I error has been fixed. For more complex tests involving nuisance parameters, we recover the classical test by using Jeffreys prior on the nuisance parameters, while the prior on the hypothesized parameters can be arbitrary up to a large class. On the other hand, we show that using proper priors on the nuisance parameters results in a test with uniformly lower power than the classical test.