Bayes Factors via Savage-Dickey Supermodels

TL;DR

This paper introduces a novel method for computing Bayes Factors using Savage-Dickey Supermodels, which simplifies model selection by transforming it into a parameter estimation problem, enabling more efficient computation especially in high-dimensional settings.

Contribution

It proposes a new supermodel approach combining models or likelihoods to efficiently compute Bayes Factors via the Savage-Dickey Density Ratio, bypassing traditional evidence calculations.

Findings

Combined-likelihood approach outperforms the combined model approach in reliability.

Method works well for Gaussian linear and toy nonlinear models.

Potential for scalable high-dimensional Bayesian model selection.

Abstract

We outline a new method to compute the Bayes Factor for model selection which bypasses the Bayesian Evidence. Our method combines multiple models into a single, nested, Supermodel using one or more hyperparameters. Since the models are now nested the Bayes Factors between the models can be efficiently computed using the Savage-Dickey Density Ratio (SDDR). In this way model selection becomes a problem of parameter estimation. We consider two ways of constructing the supermodel in detail: one based on combined models, and a second based on combined likelihoods. We report on these two approaches for a Gaussian linear model for which the Bayesian evidence can be calculated analytically and a toy nonlinear problem. Unlike the combined model approach, where a standard Monte Carlo Markov Chain (MCMC) struggles, the combined-likelihood approach fares much better in providing a reliable estimate of the log-Bayes Factor. This scheme potentially opens the way to computationally efficient ways to compute Bayes Factors in high dimensions that exploit the good scaling properties of MCMC, as compared to methods such as nested sampling that fail for high dimensions.