Denoising diffusion probabilistic models are optimally adaptive to unknown low dimensionality

TL;DR

This paper proves that denoising diffusion probabilistic models (DDPMs) can adapt optimally to unknown low-dimensional structures in data, achieving nearly linear complexity in the intrinsic dimension, which explains their practical efficiency.

Contribution

The paper demonstrates that DDPMs have nearly optimal adaptive iteration complexity proportional to the intrinsic data dimension, improving theoretical understanding of their efficiency.

Findings

Iteration complexity scales nearly linearly with intrinsic dimension k

Optimal adaptivity to unknown low-dimensional data structures

Aligns with concurrent work establishing similar guarantees

Abstract

The denoising diffusion probabilistic model (DDPM) has emerged as a mainstream generative model in generative AI. While sharp convergence guarantees have been established for the DDPM, the iteration complexity is, in general, proportional to the ambient data dimension, resulting in overly conservative theory that fails to explain its practical efficiency. This has motivated the recent work Li and Yan (2024a) to investigate how the DDPM can achieve sampling speed-ups through automatic exploitation of intrinsic low dimensionality of data. We strengthen this line of work by demonstrating, in some sense, optimal adaptivity to unknown low dimensionality. For a broad class of data distributions with intrinsic dimension $k$, we prove that the iteration complexity of the DDPM scales nearly linearly with $k$, which is optimal when using KL divergence to measure distributional discrepancy. Notably, our work is closely aligned with the independent concurrent work Potaptchik et al. (2024) -- posted two weeks prior to ours -- in establishing nearly linear-$k$ convergence guarantees for the DDPM.