Applications of the divergence theorem in Bayesian inference and MaxEnt

TL;DR

This paper introduces a general identity based on the divergence theorem that relates expectations in Bayesian inference and MaxEnt, enabling information extraction without explicit partition function calculations.

Contribution

It generalizes the equipartition theorem to Bayesian and MaxEnt contexts, providing a new tool for inference and evidence estimation.

Findings

Derives a general expectation identity for Bayesian and MaxEnt models

Shows how to estimate information without computing the partition function

Demonstrates the relation's usefulness with fluctuation theorems

Abstract

Given a probability density $P({\bf x}|{\boldsymbol \lambda})$, where $\bf x$ represents continuous degrees of freedom and $\lambda$ a set of parameters, it is possible to construct a general identity relating expectations of observable quantities, which is a generalization of the equipartition theorem in Thermodynamics. In this work we explore some of the consequences of this relation, both in the context of sampling distributions and Bayesian posteriors, and how it can be used to extract some information without the need for explicit calculation of the partition function (or the Bayesian evidence, in the case of posterior expectations). Together with the general family of fluctuation theorems it constitutes a powerful tool for Bayesian/MaxEnt problems.