Linear Convergence of Diffusion Models Under the Manifold Hypothesis

TL;DR

This paper proves that diffusion models converge linearly with respect to the intrinsic dimension of the data manifold, improving understanding of their efficiency under the manifold hypothesis.

Contribution

It establishes that diffusion models require a number of steps linear in the intrinsic dimension for convergence, combining previous bounds and proving sharpness.

Findings

Convergence in KL divergence is linear in the intrinsic dimension d.

The linear dependency on d is shown to be sharp.

The results improve theoretical understanding of diffusion models' efficiency.

Abstract

Score-matching generative models have proven successful at sampling from complex high-dimensional data distributions. In many applications, this distribution is believed to concentrate on a much lower $d$-dimensional manifold embedded into $D$-dimensional space; this is known as the manifold hypothesis. The current best-known convergence guarantees are either linear in $D$ or polynomial (superlinear) in $d$. The latter exploits a novel integration scheme for the backward SDE. We take the best of both worlds and show that the number of steps diffusion models require in order to converge in Kullback-Leibler~(KL) divergence is linear (up to logarithmic terms) in the intrinsic dimension $d$. Moreover, we show that this linear dependency is sharp.