When Shannon and Khinchin meet Shore and Johnson: equivalence of information theory and statistical inference axiomatics

TL;DR

This paper unifies the axiomatic foundations of information theory and statistical inference, demonstrating their equivalence through a generalized arithmetic framework and showing they produce the same class of entropies.

Contribution

It rephrases Shannon-Khinchin axioms using generalized arithmetics and proves their equivalence with Shore-Johnson axioms, unifying the foundations of information theory and statistical inference.

Findings

Shannon-Khinchin and Shore-Johnson axioms yield identical entropy classes

Unified framework re-establishes the link between information theory and inference

Demonstrates the equivalence of entropic functionals across frameworks

Abstract

We propose a unified framework for both Shannon-Khinchin and Shore-Johnson axiomatic systems. We do it by rephrasing Shannon-Khinchine axioms in terms of generalized arithmetics of Kolmogorov and Nagumo. We prove that the two axiomatic schemes yield identical classes of entropic functionals -- Uffink class of entropies. This allows to re-establish the entropic parallelism between information theory and statistical inference that has seemed to be "broken" by the use of non-Shannonian entropies.