On Shannon-Jaynes Entropy and Fisher Information

TL;DR

This paper revisits the Maximum Entropy principle, validating Shannon-Jaynes relative entropy as optimal for probability updating and proposing a rule for minimal sensitivity to coarse-graining, with implications for physics as inference.

Contribution

It provides a theoretical justification for using Shannon-Jaynes entropy in probability updates and introduces a constructive rule for coarse-graining sensitivity.

Findings

Shannon-Jaynes entropy is optimal for probability updating.

A constructive rule minimizes sensitivity to coarse-graining.

Implications for interpreting physics laws as inference rules.

Abstract

The fundamentals of the Maximum Entropy principle as a rule for assigning and updating probabilities are revisited. The Shannon-Jaynes relative entropy is vindicated as the optimal criterion for use with an updating rule. A constructive rule is justified which assigns the probabilities least sensitive to coarse-graining. The implications of these developments for interpreting physics laws as rules of inference upon incomplete information are briefly discussed.