On Finite-Time Mutual Information

TL;DR

This paper investigates finite-time mutual information for Gaussian processes, deriving formulas, revealing the exceed-average phenomenon, and connecting it with operator theory to inform practical communication system design.

Contribution

It introduces a novel finite-time mutual information formula, reveals the exceed-average phenomenon, and links the problem to operator theory, advancing understanding beyond Shannon's asymptotic capacity.

Findings

Finite-time mutual information can exceed the average mutual information.

Derived a formula using Mercer expansion for typical autocorrelation cases.

Proved the existence of the exceed-average phenomenon and its compatibility with Shannon capacity.

Abstract

Shannon-Hartley theorem can accurately calculate the channel capacity when the signal observation time is infinite. However, the calculation of finite-time mutual information, which remains unknown, is essential for guiding the design of practical communication systems. In this paper, we investigate the mutual information between two correlated Gaussian processes within a finite-time observation window. We first derive the finite-time mutual information by providing a limit expression. Then we numerically compute the mutual information within a single finite-time window. We reveal that the number of bits transmitted per second within the finite-time window can exceed the mutual information averaged over the entire time axis, which is called the exceed-average phenomenon. Furthermore, we derive a finite-time mutual information formula under a typical signal autocorrelation case by utilizing the Mercer expansion of trace class operators, and reveal the connection between the finite-time mutual information problem and the operator theory. Finally, we analytically prove the existence of the exceed-average phenomenon in this typical case, and demonstrate its compatibility with the Shannon capacity.