Adaptivity and Convergence of Probability Flow ODEs in Diffusion Generative Models

TL;DR

This paper provides theoretical guarantees for the probability flow ODE in diffusion models, showing it can adapt to low-dimensional structures in data and achieve faster convergence rates than previous methods.

Contribution

It establishes a dimension-free convergence rate for the probability flow ODE sampler, demonstrating its ability to exploit intrinsic data structures for efficient sampling.

Findings

Achieves $O(k/T)$ convergence rate in total variation distance

Convergence rate depends on intrinsic dimension, not ambient dimension

Improves upon existing results by exploiting low-dimensional structures

Abstract

Score-based generative models, which transform noise into data by learning to reverse a diffusion process, have become a cornerstone of modern generative AI. This paper contributes to establishing theoretical guarantees for the probability flow ODE, a widely used diffusion-based sampler known for its practical efficiency. While a number of prior works address its general convergence theory, it remains unclear whether the probability flow ODE sampler can adapt to the low-dimensional structures commonly present in natural image data. We demonstrate that, with accurate score function estimation, the probability flow ODE sampler achieves a convergence rate of $O(k/T)$ in total variation distance (ignoring logarithmic factors), where $k$ is the intrinsic dimension of the target distribution and $T$ is the number of iterations. This dimension-free convergence rate improves upon existing results that scale with the typically much larger ambient dimension, highlighting the ability of the probability flow ODE sampler to exploit intrinsic low-dimensional structures in the target distribution for faster sampling.