Why bother with Bayesian t-tests?

TL;DR

This paper critically examines Bayesian t-tests, revealing they are mathematically equivalent to classical tests and often lead to misinterpretation, advocating for an alternative distributional approach to hypothesis testing.

Contribution

The paper demonstrates the equivalence of Bayesian and classical t-tests and proposes an alternative distributional method addressing their limitations.

Findings

Bayesian t-tests are mathematically equivalent to classical t-tests.

Bayesian t-tests can be misinterpreted, sometimes more problematically than classical tests.

An alternative distributional approach is proposed for hypothesis testing.

Abstract

Given the well-known and fundamental problems with hypothesis testing via classical (point-form) significance tests, there has been a general move to alternative approaches, often focused on the Bayesian t-test. We show that the Bayesian t-test approach does not address the observed problems with classical significance testing, that Bayesian and classical t-tests are mathematically equivalent and linearly related in order of magnitude (so that the Bayesian t-test providing no further information beyond that given by point-form significance tests), and that Bayesian t-tests are subject to serious risks of misinterpretation, in some cases more problematic than seen for classical tests (with, for example, a negative sample mean in an experiment giving strong Bayesian t-test evidence in favour of a positive population mean). We do not suggest a return to the classical, point-form significance approach to hypothesis testing. Instead we argue for an alternative distributional approach to significance testing, which addresses the observed problems with classical hypothesis testing and provides a natural link between the Bayesian and frequentist approaches.