BB plot: A Tool for Accurate Model Selection Using Bayes factors

TL;DR

The paper introduces the BB plot, a diagnostic tool based on Bayes factors, to validate and estimate background distributions in model selection, demonstrated with gravitational wave data.

Contribution

It presents the BB plot relationship as a new diagnostic for validating Bayes factor calculations and estimating background distributions efficiently.

Findings

BB plot effectively validates Bayes factor calculations in GW data.

The method estimates background distributions at low computational cost.

Applied to GW lensing, it bounds the significance of GW231123.

Abstract

A common task in physics and astronomy is studying which of the competing hypotheses the data prefer. This is usually done by computing the Bayes factor between the two hypotheses, and either interpreting it in terms of the posterior odds or as a ranking statistic for a frequentist p-value test. Here we describe a relationship between the Bayes factor and its distributions under the two competing hypotheses, called the Bayes factor-Bayes factor (BB) relationship, expressed as a diagnostic plot. Using examples from gravitational wave (GW) astronomy, we demonstrate how the BB plot can validate the accuracy of Bayes factor calculations. The BB relationship may also be useful for estimating background distributions of the Bayes factor at low computational cost, even analytically in some cases. We apply this technique in the context of wave-optics lensing of GWs, extrapolating the background distribution from GWTC4 to put a rough bound of $\lesssim 4.1 \sigma$ on the statistical significance of GW231123.