Approximating evidence via bounded harmonic means

TL;DR

This paper introduces ECMLE, a novel estimator for Bayesian model evidence that overcomes the variance issues of traditional harmonic mean estimators by using elliptical coverings of high posterior density regions, improving stability and accuracy.

Contribution

The paper proposes ECMLE, an innovative evidence approximation method utilizing elliptical coverings of HPD regions, which eliminates infinite variance and performs well in multimodal scenarios.

Findings

ECMLE outperforms recent methods like THAMES in accuracy.

ECMLE demonstrates lower variance and more stable estimates.

The method is effective in multimodal and challenging settings.

Abstract

Efficient Bayesian model selection relies on the model evidence or marginal likelihood, whose computation often requires evaluating an intractable integral. The harmonic mean estimator (HME) has long been a standard method of approximating the evidence. While computationally simple, the version introduced by Newton and Raftery (1994) potentially suffers from infinite variance. To overcome this issue,Gelfand and Dey (1994) defined a standardized representation of the estimator based on an instrumental function and Robert and Wraith (2009) later proposed to use higher posterior density (HPD) indicators as instrumental functions. Following this approach, a practical method is proposed, based on an elliptical covering of the HPD region with non-overlapping ellipsoids. The resulting estimator, called the Elliptical Covering Marginal Likelihood Estimator (ECMLE), not only eliminates the infinite-variance issue of the original HME and allows exact volume computations, but is also able to be used in multimodal settings. Through several examples, we illustrate that ECMLE outperforms other recent methods such as THAMES and its improved version (Metodiev et al 2024, 2025). Moreover, ECMLE demonstrates lower variance, a key challenge that subsequent HME variants have sought to address, and provides more stable evidence approximations, even in challenging settings.