Fast, Accurate Manifold Denoising by Tunneling Riemannian Optimization

TL;DR

This paper introduces a novel framework for efficient manifold denoising by learning to optimize directly on noisy data, combining online learning and mixed-order methods to achieve near-optimal results with improved efficiency.

Contribution

The work proposes a new test-time denoising approach that learns to optimize on the manifold of clean signals using only noisy samples, with theoretical guarantees of global optimality.

Findings

Significantly improved complexity-performance tradeoffs over nearest neighbor search.

Theoretical analysis confirms the efficiency and near-optimal denoising performance.

Experimental results on scientific manifolds demonstrate superior denoising capabilities.

Abstract

Learned denoisers play a fundamental role in various signal generation (e.g., diffusion models) and reconstruction (e.g., compressed sensing) architectures, whose success derives from their ability to leverage low-dimensional structure in data. Existing denoising methods, however, either rely on local approximations that require a linear scan of the entire dataset or treat denoising as generic function approximation problems, often sacrificing efficiency and interpretability. We consider the problem of efficiently denoising a new noisy data point sampled from an unknown $d$-dimensional manifold $M \in \mathbb{R}^D$, using only noisy samples. This work proposes a framework for test-time efficient manifold denoising, by framing the concept of "learning-to-denoise" as "learning-to-optimize". We have two technical innovations: (i) online learning methods which learn to optimize over the manifold of clean signals using only noisy data, effectively "growing" an optimizer one sample at a time. (ii) mixed-order methods which guarantee that the learned optimizers achieve global optimality, ensuring both efficiency and near-optimal denoising performance. We corroborate these claims with theoretical analyses of both the complexity and denoising performance of mixed-order traversal. Our experiments on scientific manifolds demonstrate significantly improved complexity-performance tradeoffs compared to nearest neighbor search, which underpins existing provable denoising approaches based on exhaustive search.