Information, Estimation, and Lookahead in the Gaussian channel

TL;DR

This paper investigates the relationship between mutual information and MMSE with lookahead in Gaussian channels, revealing that mutual information alone does not determine MMSE for finite lookahead, and provides explicit characterizations for Ornstein-Uhlenbeck processes.

Contribution

It establishes that mutual information rate does not fully determine MMSE with finite lookahead and characterizes MMSE behavior for Ornstein-Uhlenbeck processes and their mixtures.

Findings

Mutual information rate does not determine MMSE with finite lookahead.

MMSE with lookahead converges exponentially to non-causal error.

Explicit MMSE characterizations for Ornstein-Uhlenbeck processes.

Abstract

We consider mean squared estimation with lookahead of a continuous-time signal corrupted by additive white Gaussian noise. We show that the mutual information rate function, i.e., the mutual information rate as function of the signal-to-noise ratio (SNR), does not, in general, determine the minimum mean squared error (MMSE) with fixed finite lookahead, in contrast to the special cases with 0 and infinite lookahead (filtering and smoothing errors), respectively, which were previously established in the literature. We also establish a new expectation identity under a generalized observation model where the Gaussian channel has an SNR jump at $t=0$, capturing the tradeoff between lookahead and SNR. Further, we study the class of continuous-time stationary Gauss-Markov processes (Ornstein-Uhlenbeck processes) as channel inputs, and explicitly characterize the behavior of the minimum mean squared error (MMSE) with finite lookahead and signal-to-noise ratio (SNR). The MMSE with lookahead is shown to converge exponentially rapidly to the non-causal error, with the exponent being the reciprocal of the non-causal error. We extend our results to mixtures of Ornstein-Uhlenbeck processes, and use the insight gained to present lower and upper bounds on the MMSE with lookahead for a class of stationary Gaussian input processes, whose spectrum can be expressed as a mixture of Ornstein-Uhlenbeck spectra.