Recursive Pathways to Marginal Likelihood Estimation with Prior-Sensitivity Analysis

TL;DR

This paper explores recursive marginal likelihood estimators in Bayesian analysis, demonstrating their efficiency and flexibility through numerical examples, and introduces new heuristics and connections to nested sampling.

Contribution

It introduces a novel heuristic for assessing the importance of the bridging sequence and links recursive estimators to nested sampling techniques.

Findings

Recursive estimators are versatile for prior sensitivity analysis.

The proposed heuristic helps evaluate the bridging sequence's role.

Connections to nested sampling enhance understanding of the estimators.

Abstract

We investigate the utility to computational Bayesian analyses of a particular family of recursive marginal likelihood estimators characterized by the (equivalent) algorithms known as "biased sampling" or "reverse logistic regression" in the statistics literature and "the density of states" in physics. Through a pair of numerical examples (including mixture modeling of the well-known galaxy data set) we highlight the remarkable diversity of sampling schemes amenable to such recursive normalization, as well as the notable efficiency of the resulting pseudo-mixture distributions for gauging prior sensitivity in the Bayesian model selection context. Our key theoretical contributions are to introduce a novel heuristic ("thermodynamic integration via importance sampling") for qualifying the role of the bridging sequence in this procedure and to reveal various connections between these recursive estimators and the nested sampling technique.