Bayesian Evidence and Model Selection

TL;DR

This paper reviews Bayesian evidence and Bayes factors for model selection, discussing various computational techniques and their applications in signal processing and scientific modeling, with insights from statistical physics.

Contribution

It provides a comprehensive overview of Bayesian model selection methods, including recent variants and their practical applications across multiple scientific domains.

Findings

Comparison of analytic and numerical techniques for Bayesian evidence calculation

Application of Bayesian model testing in signal processing and scientific models

Insights from statistical physics to improve Bayesian methods

Abstract

In this paper we review the concepts of Bayesian evidence and Bayes factors, also known as log odds ratios, and their application to model selection. The theory is presented along with a discussion of analytic, approximate and numerical techniques. Specific attention is paid to the Laplace approximation, variational Bayes, importance sampling, thermodynamic integration, and nested sampling and its recent variants. Analogies to statistical physics, from which many of these techniques originate, are discussed in order to provide readers with deeper insights that may lead to new techniques. The utility of Bayesian model testing in the domain sciences is demonstrated by presenting four specific practical examples considered within the context of signal processing in the areas of signal detection, sensor characterization, scientific model selection and molecular force characterization.