The maximum relative entropy principle

TL;DR

This paper reveals that the maximum entropy principle's naive application can depend on the description level, and proposes using relative entropy with a reference distribution for invariance, demonstrated in classical, quantum, and ecological contexts.

Contribution

It introduces the correct approach of maximizing relative entropy with a reference distribution to ensure invariance under coarse-graining.

Findings

Naive maximum entropy can depend on description level

Using relative entropy with a reference ensures invariance

Applications shown in classical, quantum, and ecological systems

Abstract

We show that the naive application of the maximum entropy principle can yield answers which depend on the level of description, i.e. the result is not invariant under coarse-graining. We demonstrate that the correct approach, even for discrete systems, requires maximization of the relative entropy with a suitable reference probability, which in some instances can be deduced from the symmetry properties of the dynamics. We present simple illustrations of this crucial yet surprising feature in examples of classical and quantum statistical mechanics, as well as in the field of ecology.