Nonlinear Information Theory: Characterizing Distributional Uncertainty in Communication Models with Sublinear Expectation

TL;DR

This paper introduces a nonlinear information theory framework to analyze communication systems with uncertain, nonstationary distributions, extending classical concepts to account for distributional uncertainty using sublinear expectations.

Contribution

It develops fundamental nonlinear information measures and establishes source and channel coding theorems under distributional uncertainty, advancing the theoretical understanding of complex communication networks.

Findings

Defines nonlinear entropy, joint entropy, conditional entropy, and mutual information.

Proposes a nonlinear source coding theorem with bounds on achievable rates.

Establishes a nonlinear channel coding theorem and a rate-distortion theorem for uncertain sources.

Abstract

A mathematical framework for information-theoretic analysis is established, with a new viewpoint of describing transmitted messages and communication channels by the nonlinear expectation theory, beyond the framework of classical probability theory. The major motivation of this research is to emphasize the probabilistic distribution uncertainty within the ever increasingly complex communication networks, where random phenomena are often nonstationary, heterogeneous, and cannot be characterized by a single probability distribution. Based on the nonlinear expectation theory, in this paper we first explicitly define several fundamental concepts, such as nonlinear information entropy, nonlinear joint entropy, nonlinear conditional entropy and nonlinear mutual information, and establish their basic properties. Secondly, by using the strong law of large numbers under sublinear expectations, we propose a nonlinear source coding theorem, which shows that the nonlinear information entropy is the upper bound of the achievable coding rate of sources whose distributions are uncertain under the maximum error probability criterion, and determines a cluster point of the coding rate of such sources under the minimum error probability criterion. Thirdly, we propose a nonlinear channel coding theorem, which gives the explicit expression of the upper bound under the maximum error probability criterion and a cluster point under the minimum error probability criterion, respectively, for the achievable coding rate of communication channels whose distributions are uncertain. Additionally, we propose a nonlinear rate-distortion source coding theorem, proving that the rate distortion function based on the nonlinear mutual information is a cluster point of the lossy compression performance of uncertain-distribution sources under the minimum expected distortion criterion.