FAST-DIPS: Adjoint-Free Analytic Steps and Hard-Constrained Likelihood Correction for Diffusion-Prior Inverse Problems

TL;DR

This paper introduces FAST-DIPS, a training-free diffusion prior-based inverse problem solver that uses an analytic, adjoint-free approach with a hard measurement constraint, achieving high efficiency and competitive results.

Contribution

It proposes a novel, training-free inverse solver that replaces iterative derivatives with a closed-form projection and an analytic step size, improving efficiency without retraining.

Findings

Achieves up to 19.5× speedup over existing methods.

Maintains competitive PSNR, SSIM, and LPIPS metrics.

Proves local optimality and descent properties of the proposed method.

Abstract

Training-free diffusion priors enable inverse-problem solvers without retraining, but for nonlinear forward operators data consistency often relies on repeated derivatives or inner optimization/MCMC loops with conservative step sizes, incurring many iterations and denoiser/score evaluations. We propose a training-free solver that replaces these inner loops with a hard measurement-space feasibility constraint (closed-form projection) and an analytic, model-optimal step size, enabling a small, fixed compute budget per noise level. Anchored at the denoiser prediction, the correction is approximated via an adjoint-free, ADMM-style splitting with projection and a few steepest-descent updates, using one VJP and either one JVP or a forward-difference probe, followed by backtracking and decoupled re-annealing. We prove local model optimality and descent under backtracking for the step-size rule, and derive an explicit KL bound for mode-substitution re-annealing under a local Gaussian conditional surrogate. We also develop a latent variant and a one-parameter pixel$\rightarrow$latent hybrid schedule. Experiments achieve competitive PSNR/SSIM/LPIPS with up to 19.5$\times$ speedup, without hand-coded adjoints or inner MCMC.