A Hybrid Approximation to the Marginal Likelihood

TL;DR

This paper introduces a hybrid method that combines probabilistic and deterministic approaches to estimate the marginal likelihood in Bayesian analysis, especially effective in high-dimensional or limited-sample scenarios.

Contribution

It proposes a novel hybrid approximation method that partitions the parameter space using MCMC samples and forms deterministic estimates over these partitions, improving efficiency and accuracy.

Findings

Effective in high-dimensional settings

Performs well with limited or approximate MCMC samples

Versatile across various examples

Abstract

Computing the marginal likelihood or evidence is one of the core challenges in Bayesian analysis. While there are many established methods for estimating this quantity, they predominantly rely on using a large number of posterior samples obtained from a Markov Chain Monte Carlo (MCMC) algorithm. As the dimension of the parameter space increases, however, many of these methods become prohibitively slow and potentially inaccurate. In this paper, we propose a novel method in which we use the MCMC samples to learn a high probability partition of the parameter space and then form a deterministic approximation over each of these partition sets. This two-step procedure, which constitutes both a probabilistic and a deterministic component, is termed a Hybrid approximation to the marginal likelihood. We demonstrate its versatility in a plethora of examples with varying dimension and sample size, and we also highlight the Hybrid approximation's effectiveness in situations where there is either a limited number or only approximate MCMC samples available.