Query Lower Bounds for Diffusion Sampling

TL;DR

This paper establishes fundamental lower bounds on the number of score evaluations needed for diffusion sampling, revealing the necessity of multiscale noise schedules in high-dimensional settings.

Contribution

It provides the first information-theoretic lower bounds on score queries for diffusion sampling, showing a fundamental $ ilde{ ext{O}}(\sqrt{d})$ complexity in high dimensions.

Findings

Any diffusion sampler requires at least $ ilde{ ext{O}}(\sqrt{d})$ score queries.

The proof explains the necessity of multiscale noise schedules in practice.

Score evaluations with polynomial accuracy are insufficient to bypass the lower bounds.

Abstract

Diffusion models generate samples by iteratively querying learned score estimates. A rapidly growing literature focuses on accelerating sampling by minimizing the number of score evaluations, yet the information-theoretic limits of such acceleration remain unclear. In this work, we establish the first score query lower bounds for diffusion sampling. We prove that for $d$-dimensional distributions, given access to score estimates with polynomial accuracy $\varepsilon=d^{-O(1)}$ (in any $L^p$ sense), any sampling algorithm requires $\widetilde{\Omega}(\sqrt{d})$ adaptive score queries. In particular, our proof shows that any sampler must search over $\widetilde{\Omega}(\sqrt{d})$ distinct noise levels, providing a formal explanation for why multiscale noise schedules are necessary in practice.