Entropy under disintegrations

TL;DR

This paper generalizes the concept of differential entropy for measures on arbitrary measurable spaces, establishing an asymptotic equipartition property and a chain rule for disintegrations, with applications to Haar measures.

Contribution

It extends entropy concepts to general measure spaces, providing a novel presentation of the asymptotic equipartition property and a chain rule for disintegrations.

Findings

Established asymptotic equipartition property in general measure spaces.

Derived a chain rule for entropies under measure disintegrations.

Applied results to Haar measures in canonical relations.

Abstract

We consider the differential entropy of probability measures absolutely continuous with respect to a given $\sigma$-finite reference measure on an arbitrary measurable space. We state the asymptotic equipartition property in this general case; the result is part of the folklore but our presentation is to some extent novel. Then we study a general framework under which such entropies satisfy a chain rule: disintegrations of measures. We give an asymptotic interpretation for conditional entropies in this case. Finally, we apply our result to Haar measures in canonical relation.