A Sharp KL-Convergence Analysis for Diffusion Models under Minimal Assumptions

TL;DR

This paper provides a refined analysis of diffusion models' convergence, achieving better dependence on error tolerance and data dimension under minimal assumptions, thus improving the efficiency of sampling algorithms.

Contribution

The authors introduce a novel analysis that improves the step complexity for KL divergence convergence in diffusion models without smoothness assumptions.

Findings

Reduced the number of steps to achieve target KL divergence from previous bounds.

Established linear dependence on data dimension for discretization error.

Provided a new analysis framework for diffusion model convergence under minimal assumptions.

Abstract

Diffusion-based generative models have emerged as highly effective methods for synthesizing high-quality samples. Recent works have focused on analyzing the convergence of their generation process with minimal assumptions, either through reverse SDEs or Probability Flow ODEs. The best known guarantees, without any smoothness assumptions, for the KL divergence so far achieve a linear dependence on the data dimension $d$ and an inverse quadratic dependence on $\varepsilon$. In this work, we present a refined analysis that improves the dependence on $\varepsilon$. We model the generation process as a composition of two steps: a reverse ODE step, followed by a smaller noising step along the forward process. This design leverages the fact that the ODE step enables control in Wasserstein-type error, which can then be converted into a KL divergence bound via noise addition, leading to a better dependence on the discretization step size. We further provide a novel analysis to achieve the linear $d$-dependence for the error due to discretizing this Probability Flow ODE in absence of any smoothness assumptions. We show that $\tilde{O}\left(\tfrac{d\log^{3/2}(\frac{1}{\delta})}{\varepsilon}\right)$ steps suffice to approximate the target distribution corrupted with Gaussian noise of variance $\delta$ within $O(\varepsilon^2)$ in KL divergence, improving upon the previous best result, requiring $\tilde{O}\left(\tfrac{d\log^2(\frac{1}{\delta})}{\varepsilon^2}\right)$ steps.