Information Theory for Expectation Measures

TL;DR

This paper explores the application of expectation measures in information theory, addressing deterministic data, point processes, and variable sample sizes to clarify when probability measures are appropriate.

Contribution

It introduces the use of expectation measures in information theory, extending traditional concepts to deterministic data and non-fixed sample sizes.

Findings

Expectation measures help distinguish cases where probability measures are suitable.

The study clarifies the role of Kraft's Inequality with expectation measures.

Application to point processes and sampling expands the scope of information theory.

Abstract

Shannon based his information theory on the notion of probability measures as it we developed by Kolmogorov. In this paper we study some fundamental problems in information theory based on expectation measures. In the theory of expectation measures it is natural to study data sets where no randomness is present and it is also natural to study information theory for point processes as well as sampling where the sample size is not fixed. Expectation measures in combination with Kraft's Inequality can be used to clarify in which cases probability measures can be used to quantify randomness.