Unnormalized Measures in Information Theory

TL;DR

This paper explores the use of unnormalized measures in information theory, proposing the Poisson interpretation as an alternative to probability measures, leading to simplified problems and new insights across various applications.

Contribution

It introduces the Poisson interpretation for unnormalized measures, offering a novel perspective that simplifies analysis and improves methods in information theory and related fields.

Findings

Unnormalized measures can be interpreted via the Poisson perspective.

Simplified algorithms and improved inequalities are achieved.

Enhanced understanding of quantum systems through this approach.

Abstract

Information theory is built on probability measures and by definition a probability measure has total mass 1. Probability measures are used to model uncertainty, and one may ask how important it is that the total mass is one. We claim that the main reason to normalize measures is that probability measures are related to codes via Kraft's inequality. Using a minimum description length approach to statistics we will demonstrate with that measures that are not normalized require a new interpretation that we will call the Poisson interpretation. With the Poisson interpretation many problems can be simplified. The focus will shift from from probabilities to mean values. We give examples of improvements of test procedures, improved inequalities, simplified algorithms, new projection results, and improvements in our description of quantum systems.