Distributional Shrinkage I: Universal Denoiser Beyond Tweedie's Formula

TL;DR

This paper introduces universal denoisers that improve distributional recovery accuracy beyond Tweedie's formula by leveraging optimal transport and higher-order moment matching, without prior knowledge of the noise distribution.

Contribution

It develops denoisers that achieve significantly better accuracy in recovering the signal distribution from noisy observations, surpassing traditional Bayesian methods like Tweedie's formula.

Findings

Denoisers achieve $O(\sigma^4)$ and $O(\sigma^6)$ accuracy in distributional matching.

Proposed methods outperform Bayes-optimal denoisers in distributional recovery.

Denoisers can be efficiently implemented via score matching and approximate the Monge--Ampère equation.

Abstract

We study the problem of denoising when only the noise level is known, not the noise distribution. Independent noise $Z$ corrupts a signal $X$, yielding the observation $Y = X + \sigma Z$ with known $\sigma \in (0,1)$. We propose \emph{universal} denoisers, agnostic to both signal and noise distributions, that recover the signal distribution $P_X$ from $P_Y$. When the focus is on distributional recovery of $P_X$ rather than on individual realizations of $X$, our denoisers achieve order-of-magnitude improvements over the Bayes-optimal denoiser derived from Tweedie's formula, which achieves $O(\sigma^2)$ accuracy. They shrink $P_Y$ toward $P_X$ with $O(\sigma^4)$ and $O(\sigma^6)$ accuracy in matching generalized moments and densities. Drawing on optimal transport theory, our denoisers approximate the Monge--Amp\`ere equation with higher-order accuracy and can be implemented efficiently via score matching. Let $q$ denote the density of $P_Y$. For distributional denoising, we propose replacing the Bayes-optimal denoiser, $$\mathbf{T}^*(y) = y + \sigma^2 \nabla \log q(y),$$ with denoisers exhibiting less-aggressive distributional shrinkage, $$\mathbf{T}_1(y) = y + \frac{\sigma^2}{2} \nabla \log q(y),$$ $$\mathbf{T}_2(y) = y + \frac{\sigma^2}{2} \nabla \log q(y) - \frac{\sigma^4}{8} \nabla \!\left( \frac{1}{2} \| \nabla \log q(y) \|^2 + \nabla \cdot \nabla \log q(y) \right)\!.$$