An Information Theoretic Proof of the Radon-Nikodym Theorem

TL;DR

This paper provides an information theoretic proof of the Radon-Nikodym theorem, highlighting its importance in defining key concepts like Shannon entropy and f-divergences, and addressing its omission in probability and information theory texts due to proof complexity.

Contribution

It offers a novel proof of the Radon-Nikodym theorem using information theory, making the proof more accessible and emphasizing its foundational role in information measures.

Findings

Provides an information theoretic proof of the Radon-Nikodym theorem

Clarifies the theorem's role in defining Shannon entropy and divergences

Addresses the proof's complexity in traditional texts

Abstract

The Radon-Nikodym theorem plays a significant role in the definition of Shannon entropy, f-divergences, and other basic quantities in information theory. The existence of Radon Nikodym derivates appear in many text books in measure theory but in text books on probability or information theory it is often omitted because the proof is often considered to be too difficult.