The information loss of a stochastic map

TL;DR

This paper extends the concept of information loss in stochastic maps, introducing semi-functorial measures and an entropic Bayes' rule within a categorical framework, providing new insights into conditional entropy.

Contribution

It introduces a stochastic extension of the Baez-Fritz-Leinster characterization, semi-functorial information measures, and an entropic Bayes' rule applicable in Markov categories.

Findings

Defines semi-functorial information measures.

Characterizes conditional entropy via an entropic Bayes' rule.

Recovers conditional entropy as a key measure in stochastic maps.

Abstract

We provide a stochastic extension of the Baez-Fritz-Leinster characterization of the Shannon information loss associated with a measure-preserving function. This recovers the conditional entropy and a closely related information-theoretic measure that we call conditional information loss. Although not functorial, these information measures are semi-functorial, a concept we introduce that is definable in any Markov category. We also introduce the notion of an entropic Bayes' rule for information measures, and we provide a characterization of conditional entropy in terms of this rule.