Geometry-Aware Discretization Error of Diffusion Models

TL;DR

This paper derives explicit formulas for discretization errors in diffusion models, revealing how data geometry and diffusion parameters influence sampling accuracy, and guiding more effective parameter choices.

Contribution

It introduces geometry-aware discretization error formulas for diffusion models, enabling better parameter optimization based on data geometry and diffusion dynamics.

Findings

Discretization error formulas depend on data covariance spectrum.

Error adapts to data geometry and diffusion parameters.

Formulas are robust across different datasets and sampling tasks.

Abstract

Practical diffusion sampling is a numerical approximation problem: under a fixed inference budget, one must simulate a reverse-time ODE or SDE using only a limited number of denoising steps, so discretization error is often the dominant source of error. Existing non-asymptotic analyses provide convergence guarantees, but are typically too loose and too insensitive to diffusion parameters to guide practical design: broad families of schedules receive the same rates, which depend on coarse worst-case quantities such as the dimension or the drift Lipschitz constant. We take a less ambitious but more informative route. In the exact-score setting, we derive first-order asymptotic expansions of the Euler-Maruyama weak and Fr\'echet discretization errors. These formulas hold for general smooth reverse diffusions and become fully explicit under Gaussian data. They show how discretization error adapts to the geometry of the data through the covariance spectrum, and how this geometry interacts with key diffusion parameters, including the diffusion schedules and the diffusion-term coefficient. This yields tractable objectives for geometry-aware parameter optimization. Finally, we show that the qualitative predictions of the Gaussian formulas remain robust across diffusion sampling problems with different geometries, including image generation on different datasets and image posterior sampling.