High-accuracy and dimension-free sampling with diffusions

TL;DR

This paper introduces a novel diffusion model solver that achieves high-accuracy sampling with iteration complexity scaling polylogarithmically in inverse accuracy, independent of ambient dimension, improving efficiency in high-dimensional settings.

Contribution

The authors develop a new diffusion solver using low-degree approximation and collocation, providing the first high-accuracy guarantees with dimension-free complexity dependence.

Findings

Iteration complexity scales polylogarithmically with 1/ε

Complexity depends on effective radius, not ambient dimension

Achieves high-accuracy sampling with fewer iterations

Abstract

Diffusion models have shown remarkable empirical success in sampling from rich multi-modal distributions. Their inference relies on numerically solving a certain differential equation. This differential equation cannot be solved in closed form, and its resolution via discretization typically requires many small iterations to produce \emph{high-quality} samples. More precisely, prior works have shown that the iteration complexity of discretization methods for diffusion models scales polynomially in the ambient dimension and the inverse accuracy $1/\varepsilon$. In this work, we propose a new solver for diffusion models relying on a subtle interplay between low-degree approximation and the collocation method (Lee, Song, Vempala 2018), and we prove that its iteration complexity scales \emph{polylogarithmically} in $1/\varepsilon$, yielding the first ``high-accuracy'' guarantee for a diffusion-based sampler that only uses (approximate) access to the scores of the data distribution. In addition, our bound does not depend explicitly on the ambient dimension; more precisely, the dimension affects the complexity of our solver through the \emph{effective radius} of the support of the target distribution only.