fbst: An R package for the Full Bayesian Significance Test for testing a sharp null hypothesis against its alternative via the e-value

TL;DR

The paper introduces the fbst R package implementing the Full Bayesian Significance Test (FBST), providing a Bayesian alternative to traditional NHST and p-values for hypothesis testing in psychology and cognitive sciences.

Contribution

It offers a practical implementation of FBST in R, applicable to any Bayesian model with numerical posterior, and demonstrates its use through examples in cognitive science.

Findings

The package computes the e-value, a Bayesian measure of evidence against the null hypothesis.

It provides p-values and visualizations for interpreting results.

The examples show how FBST can be applied in real research scenarios.

Abstract

Hypothesis testing is a central statistical method in psychology and the cognitive sciences. However, the problems of null hypothesis significance testing (NHST) and p-values have been debated widely, but few attractive alternatives exist. This article introduces the fbst R package, which implements the Full Bayesian Significance Test (FBST) to test a sharp null hypothesis against its alternative via the e-value. The statistical theory of the FBST has been introduced by Pereira et al. (1999) more than two decades ago and since then, the FBST has shown to be a Bayesian alternative to NHST and p-values with both theoretical and practical highly appealing properties. The algorithm provided in the fbst package is applicable to any Bayesian model as long as the posterior distribution can be obtained at least numerically. The core function of the package provides the Bayesian evidence against the null hypothesis, the e-value. Additionally, p-values based on asymptotic arguments can be computed and rich visualisations for communication and interpretation of the results can be produced. Three examples of frequently used statistical procedures in the cognitive sciences are given in this paper which demonstrate how to apply the FBST in practice using the fbst package. Based on the success of the FBST in statistical science, the fbst package should be of interest to a broad range of researchers in psychology and the cognitive sciences and hopefully will encourage researchers to consider the FBST as a possible alternative when conducting hypothesis tests of a sharp null hypothesis.