Polynomial Approximations of Conditional Expectations in Scalar Gaussian Channels

TL;DR

This paper characterizes when the MMSE estimator in a Gaussian channel is polynomial, showing it is only polynomial in trivial cases, and provides bounds on the approximation error of the conditional expectation by polynomials.

Contribution

It proves that the MMSE estimator is polynomial only for Gaussian or constant inputs and derives bounds on polynomial approximation errors for certain distributions.

Findings

MMSE estimator is polynomial only if X is Gaussian or constant

Higher derivatives of the conditional expectation are polynomial in certain functions

Polynomial approximation error decays faster than any polynomial for specific distributions

Abstract

We consider a channel $Y=X+N$ where $X$ is a random variable satisfying $\mathbb{E}[|X|]<\infty$ and $N$ is an independent standard normal random variable. We show that the minimum mean-square error estimator of $X$ from $Y,$ which is given by the conditional expectation $\mathbb{E}[X \mid Y],$ is a polynomial in $Y$ if and only if it is linear or constant; these two cases correspond to $X$ being Gaussian or a constant, respectively. We also prove that the higher-order derivatives of $y \mapsto \mathbb{E}[X \mid Y=y]$ are expressible as multivariate polynomials in the functions $y \mapsto \mathbb{E}\left[ \left( X - \mathbb{E}[X \mid Y] \right)^k \mid Y = y \right]$ for $k\in \mathbb{N}.$ These expressions yield bounds on the $2$-norm of the derivatives of the conditional expectation. These bounds imply that, if $X$ has a compactly-supported density that is even and decreasing on the positive half-line, then the error in approximating the conditional expectation $\mathbb{E}[X \mid Y]$ by polynomials in $Y$ of degree at most $n$ decays faster than any polynomial in $n.$