Provable Acceleration for Diffusion Models under Minimal Assumptions

TL;DR

This paper introduces a training-free acceleration method for diffusion models that provably improves sampling efficiency under minimal assumptions, reducing iteration complexity compared to standard methods.

Contribution

It presents a novel theoretical acceleration scheme for stochastic samplers that does not depend on restrictive distribution assumptions or higher-order score estimates.

Findings

Achieves $ ilde{O}(d^{5/4}/\sqrt{\varepsilon})$ iteration complexity for $\varepsilon$-accuracy.

Improves upon the standard $ ilde{O}(d/\varepsilon)$ complexity for small $\varepsilon$.

Does not rely on restrictive assumptions on the target distribution.

Abstract

Score-based diffusion models, while achieving minimax optimality for sampling, are often hampered by slow sampling speeds due to the high computational burden of score function evaluations. Despite the recent remarkable empirical advances in speeding up the score-based samplers, theoretical understanding of acceleration techniques remains largely limited. To bridge this gap, we propose a novel training-free acceleration scheme for stochastic samplers. Under minimal assumptions -- namely, $L^2$-accurate score estimates and a finite second-moment condition on the target distribution -- our accelerated sampler provably achieves $\varepsilon$-accuracy in total variation within $\widetilde{O}(d^{5/4}/\sqrt{\varepsilon})$ iterations, thereby significantly improving upon the $\widetilde{O}(d/\varepsilon)$ iteration complexity of standard score-based samplers for $\varepsilon\leq 1/\sqrt{d}$. Notably, our convergence theory does not rely on restrictive assumptions on the target distribution or higher-order score estimation guarantees.