Consistency Regularised Gradient Flows for Inverse Problems

TL;DR

This paper introduces a unified gradient-flow framework for inverse problems using latent diffusion models, achieving state-of-the-art results with lower computational costs.

Contribution

It proposes a novel Euclidean-Wasserstein gradient-flow method that reduces neural function evaluations and avoids backpropagation through large models.

Findings

State-of-the-art performance on canonical imaging inverse problems.

Significantly reduced neural function evaluations (NFEs).

Low-NFE inference without backpropagation through autoencoders.

Abstract

Vision-Language Latent Diffusion Models (LDMs) (Rombach et al., 2022) provide powerful generative priors for inverse problems. However, existing LDM-based inverse solvers typically require a large number of neural function evaluations (NFEs) and backpropagation through large pretrained components, leading to substantial computational costs and, in some cases, degraded reconstruction quality. We propose a unified Euclidean-Wasserstein-2 gradient-flow framework that jointly performs posterior sampling and prompt optimization in the latent space through a single flow that aligns the prior and posterior with the observed data. Combined with few-step latent text-to-image models, this formulation enables low-NFE inference without backpropagation through autoencoders. Experiments across several canonical imaging inverse problems show that our method achieves state-of-the-art performance with significantly reduced computational cost.