Warm Starts Accelerate Conditional Diffusion

TL;DR

Warm-Start Diffusion (WSD) accelerates conditional generative models by providing an informed deterministic initial guess, significantly reducing the number of steps needed for high-quality sample generation.

Contribution

The paper introduces WSD, a simple method that uses a deterministic informed prior to speed up diffusion-based generative models for conditional tasks.

Findings

WSD reduces diffusion steps from hundreds to 4-6 for realistic samples.

WSD outperforms standard diffusion in efficient sampling regimes.

Performance saturates at around 10-12 steps with WSD.

Abstract

Generative models like diffusion and flow-matching create high-fidelity samples by progressively refining noise. The refinement process is notoriously slow, often requiring hundreds of function evaluations. We introduce Warm-Start Diffusion (WSD), a method that uses a simple, deterministic model to dramatically accelerate conditional generation by providing a better starting point. Instead of starting generation from an uninformed $N(\boldsymbol{0}, I)$ prior, our deterministic warm-start model predicts an informed prior $N(\hat{\boldsymbol{\mu}}_C, \text{diag}(\hat{\boldsymbol{\sigma}}^2_C))$, whose moments are conditioned on the input context $C$. This warm start substantially reduces the distance the generative process must traverse, and therefore the number of diffusion steps required, particularly when the context $C$ is strongly informative. WSD is applicable to any standard diffusion or flow matching algorithm, is orthogonal to and synergistic with other fast sampling techniques like efficient solvers, and is simple to implement. We test WSD in a variety of settings, and find that it substantially outperforms standard diffusion in the efficient sampling regime, generating realistic samples using only 4-6 function evaluations, and saturating performance with 10-12.