Model-free filtering in high dimensions via projection and score-based diffusions

TL;DR

This paper introduces a novel, model-free approach for denoising high-dimensional signals lying on low-dimensional manifolds, utilizing score-based diffusion models and score matching to estimate the metric projection from noisy observations.

Contribution

It develops a new score-based diffusion method for manifold denoising that does not rely on explicit model assumptions, with theoretical guarantees in high dimensions.

Findings

Posterior concentrates near the true projection in high dimensions

Score matching effectively estimates the Bayesian posterior in the diffusion model

Method achieves model-free denoising on low-dimensional manifolds

Abstract

We consider the problem of recovering a latent signal $X$ from its noisy observation $Y$. The unknown law $\mathbb{P}^X$ of $X$, and in particular its support $\mathscr{M}$, are accessible only through a large sample of i.i.d.\ observations. We further assume $\mathscr{M}$ to be a low-dimensional submanifold of a high-dimensional Euclidean space $\mathbb{R}^d$. As a filter or denoiser $\widehat X$, we suggest an estimator of the metric projection $\pi_{\mathscr{M}}(Y)$ of $Y$ onto the manifold $\mathscr{M}$. To compute this estimator, we study an auxiliary semiparametric model in which $Y$ is obtained by adding isotropic Laplace noise to $X$. Using score matching within a corresponding diffusion model, we obtain an estimator of the Bayesian posterior $\mathbb{P}^{X \mid Y}$ in this setup. Our main theoretical results show that, in the limit of high dimension $d$, this posterior $\mathbb{P}^{X\mid Y}$ is concentrated near the desired metric projection $\pi_{\mathscr{M}}(Y)$.