Conditional maximum-entropy method for selecting prior distributions in Bayesian statistics

TL;DR

The paper introduces the C-MaxEnt method, a novel approach inspired by statistical mechanics, for selecting prior distributions in Bayesian parameter estimation, which simplifies prior calculation and uncovers new insights beyond Jeffreys's rules.

Contribution

It presents a new conditional maximum-entropy method for prior selection in Bayesian statistics, linking statistical mechanics principles to prior derivation.

Findings

Yields Jeffreys's rules from the method

Provides a simple way to calculate priors from likelihoods

Reveals new structures behind traditional priors

Abstract

The conditional maximum-entropy method (abbreviated here as C-MaxEnt) is formulated for selecting prior probability distributions in Bayesian statistics for parameter estimation. This method is inspired by a statistical-mechanical approach to systems governed by dynamics with largely-separated time scales and is based on three key concepts: conjugate pairs of variables, dimensionless integration measures with coarse-graining factors and partial maximization of the joint entropy. The method enables one to calculate a prior purely from a likelihood in a simple way. It is shown in particular how it not only yields Jeffreys's rules but also reveals new structures hidden behind them.