Using Statistical Mechanics to Improve Real-World Bayesian Inference: A New Method Combining Tempered Posteriors and Wang-Landau Sampling

TL;DR

This paper introduces a novel Bayesian inference method that leverages statistical mechanics concepts, specifically tempered posteriors and Wang-Landau sampling, to enhance predictive accuracy with less effort.

Contribution

It reformulates Bayesian inference using statistical mechanics, combining tempered posteriors and Wang-Landau sampling to optimize posterior distributions efficiently.

Findings

Effective in real-world materials science problem with complex data

Identifies transition temperature for optimal predictive performance

Reduces computational and human effort in Bayesian inference

Abstract

We present a simple method to obtain optimal posterior distributions and improve the quality of Bayesian inference with reduced human and computational effort. Bayes' Theorem is reformulated in the language of statistical mechanics, wherein an improved posterior -- referred to as a tempered posterior -- is defined analogously to a canonical probability distribution at temperature $\tau$. Wang-Landau sampling is used to obtain the density of states of the posterior probability, and signals analogous to those of phase transitions are extracted from a single simulation. In addition, the transition temperature is easily identified, providing the tempered posterior with optimal predictive performance. We demonstrate the efficacy of the method on a real-world problem in materials science (equation of state modeling) with messy data, a high-dimensional and correlated input parameter space, and "frustration" among model outputs.