Maxallent: Maximizers of all Entropies and Uncertainty of Uncertainty

TL;DR

This paper introduces a unified framework for understanding and comparing various entropy measures through the concept of the Markov order, highlighting the inherent uncertainty in selecting the optimal entropy for different applications.

Contribution

It develops a general ordering of distributions based on Markov processes, enabling the identification of multiple entropy maximizers and addressing the 'uncertainty of uncertainty' in non-equilibrium systems.

Findings

All continuous-time Markov processes are monotonic with respect to the Markov order.

A constructive description of the set of entropy maximizers is provided.

Two decomposition theorems facilitate the analysis of Markov processes.

Abstract

The entropy maximum approach (Maxent) was developed as a minimization of the subjective uncertainty measured by the Boltzmann--Gibbs--Shannon entropy. Many new entropies have been invented in the second half of the 20th century. Now there exists a rich choice of entropies for fitting needs. This diversity of entropies gave rise to a Maxent "anarchism". Maxent approach is now the conditional maximization of an appropriate entropy for the evaluation of the probability distribution when our information is partial and incomplete. The rich choice of non-classical entropies causes a new problem: which entropy is better for a given class of applications? We understand entropy as a measure of uncertainty which increases in Markov processes. In this work, we describe the most general ordering of the distribution space, with respect to which all continuous-time Markov processes are monotonic (the Markov order). For inference, this approach results in a set of conditionally "most random" distributions. Each distribution from this set is a maximizer of its own entropy. This "uncertainty of uncertainty" is unavoidable in analysis of non-equilibrium systems. Surprisingly, the constructive description of this set of maximizers is possible. Two decomposition theorems for Markov processes provide a tool for this description.