The distribution of Bayes' ratio

TL;DR

This paper introduces a new frequentist approach to Bayesian evidence ratios, allowing for model rejection based on sampling distributions, reducing prior dependence, and replacing Jeffrey's scale with probability thresholds, with applications in cosmology.

Contribution

The paper develops the FB method, transforming Bayesian evidence ratios into frequentist statistics, enabling model rejection and prior independence in cosmological data analysis.

Findings

The FB method allows rejection of poor models based on sampling distributions.

The approach reduces prior dependence in evidence ratios with weak priors.

Application to supernovae Ia data demonstrates practical utility.

Abstract

The ratio of Bayesian evidences is a popular tool in cosmology to compare different models. There are however several issues with this method: Bayes' ratio depends on the prior even in the limit of non-informative priors, and Jeffrey's scale, used to assess the test, is arbitrary. Moreover, the standard use of Bayes' ratio is often criticized for being unable to reject models. In this paper, we address these shortcoming by promoting evidences and evidence ratios to frequentist statistics and deriving their sampling distributions. By comparing the evidence ratios to their sampling distributions, poor fitting models can now be rejected. Our method additionally does not depend on the prior in the limit of very weak priors, thereby safeguarding the experimenter against premature rejection of a theory with a uninformative prior, and replaces the arbitrary Jeffrey's scale by probability thresholds for rejection. We provide analytical solutions for some simplified cases (Gaussian data, linear parameters, and nested models), and we apply the method to cosmological supernovae Ia data. We dub our method the FB method, for Frequentist-Bayesian.