Monte Carlo Approximation of Bayes Factors via Mixing with Surrogate Distributions

TL;DR

This paper introduces a novel Monte Carlo method for estimating Bayes factors by mixing the target posterior with a tractable surrogate distribution, enhancing convergence and applicability across various models.

Contribution

It proposes a new approach using surrogate distributions and the Wang-Landau algorithm, with strategies for global jumps and surrogate construction, improving Bayes factor estimation.

Findings

Faster convergence achieved with momentum acceleration.

Effective in models like Log-Gaussian Cox process and Bayesian Lasso.

New jumping mechanism enhances reversible jump MCMC methods.

Abstract

By mixing the target posterior distribution with a surrogate distribution, of which the normalizing constant is tractable, we propose a method for estimating the marginal likelihood using the Wang-Landau algorithm. We show that a faster convergence of the proposed method can be achieved via the momentum acceleration. Two implementation strategies are detailed: (i) facilitating global jumps between the posterior and surrogate distributions via the Multiple-try Metropolis; (ii) constructing the surrogate via the variational approximation. When a surrogate is difficult to come by, we describe a new jumping mechanism for general reversible jump Markov chain Monte Carlo algorithms, which combines the Multiple-try Metropolis and a directional sampling algorithm. We illustrate the proposed methods on several statistical models, including the Log-Gaussian Cox process, the Bayesian Lasso, the logistic regression, and the g-prior Bayesian variable selection.