A Sharp Convergence Theory for The Probability Flow ODEs of Diffusion Models

TL;DR

This paper provides a non-asymptotic convergence analysis for the probability flow ODE sampler in diffusion models, showing nearly linear dimension dependence and how score estimation errors impact data generation quality.

Contribution

It introduces the first nearly linear dimension-dependent convergence theory for the probability flow ODE sampler with minimal assumptions on the target distribution.

Findings

D/ε iterations suffice for ε-approximation in total variation distance.

Nearly linear dimension dependence (in d) is established for the sampler.

Score estimation errors directly influence the quality of data generation.

Abstract

Diffusion models, which convert noise into new data instances by learning to reverse a diffusion process, have become a cornerstone in contemporary generative modeling. In this work, we develop non-asymptotic convergence theory for a popular diffusion-based sampler (i.e., the probability flow ODE sampler) in discrete time, assuming access to $\ell_2$-accurate estimates of the (Stein) score functions. For distributions in $\mathbb{R}^d$, we prove that $d/\varepsilon$ iterations -- modulo some logarithmic and lower-order terms -- are sufficient to approximate the target distribution to within $\varepsilon$ total-variation distance. This is the first result establishing nearly linear dimension-dependency (in $d$) for the probability flow ODE sampler. Imposing only minimal assumptions on the target data distribution (e.g., no smoothness assumption is imposed), our results also characterize how $\ell_2$ score estimation errors affect the quality of the data generation processes. In contrast to prior works, our theory is developed based on an elementary yet versatile non-asymptotic approach without the need of resorting to SDE and ODE toolboxes.