High-accuracy sampling for diffusion models and log-concave distributions

TL;DR

This paper introduces highly efficient algorithms for sampling from diffusion models and log-concave distributions, achieving exponential improvements in error and complexity under minimal assumptions.

Contribution

It provides the first $ ext{polylog}(1/ ext{error})$ complexity sampler for general log-concave distributions using only gradient evaluations.

Findings

Achieves $ ext{polylog}(1/ ext{error})$ steps for diffusion sampling with $ ilde{O}( ext{error})$-accurate score estimates.

Reduces complexity to $ ilde{O}(d_ ext{intrinsic} ext{polylog}(1/ ext{error}))$ under minimal data assumptions.

Provides the first $ ext{polylog}(1/ ext{error})$ sampler for log-concave distributions with only gradient evaluations.

Abstract

We present algorithms for diffusion model sampling which obtain $\delta$-error in $\mathrm{polylog}(1/\delta)$ steps, given access to $\widetilde O(\delta)$-accurate score estimates in $L^2$. This is an exponential improvement over all previous results. Specifically, under minimal data assumptions, the complexity is $\widetilde O(d_\star \mathrm{polylog}(1/\delta))$ where $d_\star$ is the intrinsic dimension of the data. Further, under a non-uniform $L$-Lipschitz condition, the complexity reduces to $\widetilde O(L \mathrm{polylog}(1/\delta))$. Our approach also yields the first $\mathrm{polylog}(1/\delta)$ complexity sampler for general log-concave distributions using only gradient evaluations.