Multivariate Priors and the Linearity of Optimal Bayesian Estimators under Gaussian Noise

TL;DR

This paper characterizes when the optimal Bayesian estimator under Gaussian noise is linear, showing it is only Gaussian priors for 1 ≤ p ≤ 2, and infinitely many priors for p > 2.

Contribution

It provides a complete characterization of the priors that lead to linear Bayesian estimators under Gaussian noise for different p-norm fidelity criteria.

Findings

For 1 ≤ p ≤ 2, the optimal estimator is linear only if the prior is Gaussian.

For p > 2, infinitely many priors can produce linear estimators.

The results clarify the relationship between prior distributions and estimator linearity under Gaussian noise.

Abstract

Consider the task of estimating a random vector $X$ from noisy observations $Y = X + Z$, where $Z$ is a standard normal vector, under the $L^p$ fidelity criterion. This work establishes that, for $1 \leq p \leq 2$, the optimal Bayesian estimator is linear and positive definite if and only if the prior distribution on $X$ is a (non-degenerate) multivariate Gaussian. Furthermore, for $p > 2$, it is demonstrated that there are infinitely many priors that can induce such an estimator.