An axiomatic characterization of mutual information

TL;DR

This paper provides an axiomatic characterization of mutual information, establishing it as the unique measure satisfying specific axioms, including a novel one based on Markov triangles, without using logarithms.

Contribution

It introduces a new axiom involving Markov triangles for characterizing mutual information, expanding the theoretical understanding of information measures.

Findings

Mutual information is uniquely characterized by a set of axioms.

A new axiom based on Markov triangles is proposed.

Proofs are coordinate-free, avoiding logarithmic calculations.

Abstract

We characterize mutual information as the unique map on ordered pairs of random variables satisfying a set of axioms similar to those of Faddeev's characterization of the Shannon entropy. There is a new axiom in our characterization however which has no analogue for Shannon entropy, based on the notion of a Markov triangle, which may be thought of as a composition of communication channels for which conditional entropy acts functorially. Our proofs are coordinate-free in the sense that no logarithms appear in our calculations.