Denoising of structured random processes

TL;DR

This paper introduces the Q-MAP denoiser, a computationally efficient method for denoising structured random processes, which asymptotically achieves optimal performance related to the source's information dimension, and extends to learning-based image denoising.

Contribution

The paper proposes the Q-MAP denoiser, a novel approach that simplifies source property utilization and achieves asymptotic optimality in high SNR regimes for structured sources.

Findings

Q-MAP denoiser asymptotically matches the source's information dimension.

For memoryless sources, the method achieves known optimal limits.

Initial image denoising results using learned Q-MAP on ImageNet are promising.

Abstract

Denoising stationary process $(X_i)_{i \in Z}$ corrupted by additive white Gaussian noise is a classic and fundamental problem in information theory and statistical signal processing. Despite considerable progress in designing efficient denoising algorithms, for general analog sources, theoretically-founded computationally-efficient methods are yet to be found. For instance in denoising $X^n$ corrupted by noise $Z^n$ as $Y^n=X^n+Z^n$, given the full distribution of $X^n$, a minimum mean square error (MMSE) denoiser needs to compute $E[X^n|Y^n]$. However, for general sources, computing $E[X^n|Y^n]$ is computationally very challenging, if not infeasible. In this paper, starting by a Bayesian setup, where the source distribution is fully known, a novel denoising method, namely, quantized maximum a posteriori (Q-MAP) denoiser, is proposed and its asymptotic performance in the high signal to noise ratio regime is analyzed. Both for memoryless sources, and for structured first-order Markov sources, it is shown that, asymptotically, as $\sigma$ converges to zero, ${1\over \sigma^2}E[(X_i-\hat{X}^{\rm Q-MAP}_i)^2]$ achieved by Q-MAP denoiser converges to the information dimension of the source. For the studied memoryless sources, this limit is known to be optimal. A key advantage of the Q-MAP denoiser is that, unlike an MMSE denoiser, it highlights the key properties of the source distribution that are to be used in its denoising. This property dramatically reduces the computational complexity of approximating the solution of the Q-MAP denoiser. Additionally, it naturally leads to a learning-based denoiser. Using ImageNet database for training, initial simulation results exploring the performance of such a learning-based denoiser in image denoising are presented.