Tight MMSE Bounds for the AGN Channel Under KL Divergence Constraints on the Input Distribution

TL;DR

This paper derives tight bounds on the MMSE for Gaussian noise channels when input distributions are KL-divergence constrained, identifying optimal Gaussian distributions and robust estimators, with numerical validation.

Contribution

It introduces new tight MMSE bounds under KL divergence constraints and characterizes the optimal Gaussian input distributions and estimators.

Findings

Bounds are tight and explicitly characterized.

Optimal input distributions are Gaussian with specific covariance matrices.

Numerical examples demonstrate the bounds' effectiveness.

Abstract

Tight bounds on the minimum mean square error for the additive Gaussian noise channel are derived, when the input distribution is constrained to be epsilon-close to a Gaussian reference distribution in terms of the Kullback--Leibler divergence. The distributions that attain the bounds are shown be Gaussian whose means are identical to that of the reference distribution and whose covariance matrices are defined implicitly via systems of matrix equations. The estimator that attains the upper bound is identified as a minimax optimal estimator that is robust against deviations from the assumed prior. The lower bound is shown to provide a potentially tighter alternative to the Cramer--Rao bound. Both properties are illustrated with numerical examples.