Maximum entropy, fluctuations and priors

TL;DR

This paper extends the maximum entropy method to better understand the role of entropy in ruling out distributions, with applications in thermodynamic fluctuations and Bayesian priors, providing a more exact and covariant formulation.

Contribution

It introduces a covariant, exact formulation of maximum entropy for fluctuations and derives a specific objective prior for Bayesian inference, connecting to entropic priors.

Findings

Exact, covariant formulation for thermodynamic fluctuations.

Derivation of a specific objective prior for Bayesian inference.

Connection to entropic priors in Bayesian analysis.

Abstract

The method of maximum entropy (ME) is extended to address the following problem: Once one accepts that the ME distribution is to be preferred over all others, the question is to what extent are distributions with lower entropy supposed to be ruled out. Two applications are given. The first is to the theory of thermodynamic fluctuations. The formulation is exact, covariant under changes of coordinates, and allows fluctuations of both the extensive and the conjugate intensive variables. The second application is to the construction of an objective prior for Bayesian inference. The prior obtained by following the ME method to its inevitable conclusion turns out to be a special case of what are currently known under the name of entropic priors.