Estimation Error: Distribution and Pointwise Limits

TL;DR

This paper analyzes the distribution and pointwise convergence of the estimation error in Bayesian estimation under noisy observations, extending previous $L^2$ convergence results to almost sure convergence as noise diminishes.

Contribution

It characterizes the error distribution and establishes conditions for pointwise convergence of the normalized estimation error as noise level approaches zero.

Findings

Derived the probability density functions of the estimation error and normalized error.

Identified conditions for the existence of inverse functions of conditional expectations.

Extended previous $L^2$ convergence results to almost sure convergence in estimation error analysis.

Abstract

In this paper, we examine the distribution and convergence properties of the estimation error $W = X - \hat{X}(Y)$, where $\hat{X}(Y)$ is the Bayesian estimator of a random variable $X$ from a noisy observation $Y = X +\sigma Z$ where $\sigma$ is the parameter indicating the strength of noise $Z$. Using the conditional expectation framework (that is, $\hat{X}(Y)$ is the conditional mean), we define the normalized error $\mathcal{E}_\sigma = \frac{W}{\sigma}$ and explore its properties. Specifically, in the first part of the paper, we characterize the probability density function of $W$ and $\mathcal{E}_\sigma$. Along the way, we also find conditions for the existence of the inverse functions for the conditional expectations. In the second part, we study pointwise (i.e., almost sure) convergence of $\mathcal{E}_\sigma$ as $\sigma \to 0$ under various assumptions about the noise and the underlying distributions. Our results extend some of the previous limits of $\mathcal{E}_\sigma$ as $\sigma \to 0$ studied under the $L^2$ convergence, known as the \emph{mmse dimension}, to the pointwise case.