Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models for Inverse Problems through Stochastic Contraction

TL;DR

This paper introduces CCDF, a novel sampling method for diffusion models that reduces the number of steps needed for inverse problem solutions by leveraging stochastic contraction theory and better initialization.

Contribution

The paper proposes a new sampling strategy, CCDF, which accelerates diffusion model-based inverse problem solutions by starting from a single forward diffusion and using stochastic contraction theory.

Findings

Achieves state-of-the-art results in super-resolution, inpainting, and MRI reconstruction.

Reduces sampling steps significantly compared to traditional diffusion methods.

Demonstrates theoretical justification via contraction theory.

Abstract

Diffusion models have recently attained significant interest within the community owing to their strong performance as generative models. Furthermore, its application to inverse problems have demonstrated state-of-the-art performance. Unfortunately, diffusion models have a critical downside - they are inherently slow to sample from, needing few thousand steps of iteration to generate images from pure Gaussian noise. In this work, we show that starting from Gaussian noise is unnecessary. Instead, starting from a single forward diffusion with better initialization significantly reduces the number of sampling steps in the reverse conditional diffusion. This phenomenon is formally explained by the contraction theory of the stochastic difference equations like our conditional diffusion strategy - the alternating applications of reverse diffusion followed by a non-expansive data consistency step. The new sampling strategy, dubbed Come-Closer-Diffuse-Faster (CCDF), also reveals a new insight on how the existing feed-forward neural network approaches for inverse problems can be synergistically combined with the diffusion models. Experimental results with super-resolution, image inpainting, and compressed sensing MRI demonstrate that our method can achieve state-of-the-art reconstruction performance at significantly reduced sampling steps.