Thermodynamic assessment of probability distribution divergencies and Bayesian model comparison

TL;DR

This paper introduces a thermodynamic integration approach within path sampling to estimate distribution divergences like Chernoff information, enhancing Bayesian model comparison with new estimators and a geometric perspective.

Contribution

It develops a thermodynamic framework for divergence estimation, proposes new estimators, and introduces compound paths for improved Bayesian model evaluation.

Findings

Effective estimation of distribution divergences using thermodynamic integration

Introduction of new estimators and compound paths for Bayesian evaluation

Enhanced understanding of error sources through a geometric approach

Abstract

Within path sampling framework, we show that probability distribution divergences, such as the Chernoff information, can be estimated via thermodynamic integration. The Boltzmann-Gibbs distribution pertaining to different Hamiltonians is implemented to derive tempered transitions along the path, linking the distributions of interest at the endpoints. Under this perspective, a geometric approach is feasible, which prompts intuition and facilitates tuning the error sources. Additionally, there are direct applications in Bayesian model evaluation. Existing marginal likelihood and Bayes factor estimators are reviewed here along with their stepping-stone sampling analogues. New estimators are presented and the use of compound paths is introduced.