Are First-Order Diffusion Samplers Really Slower? A Fast Forward-Value Approach

TL;DR

This paper introduces a novel first-order diffusion sampler that challenges the belief that higher-order methods are always faster, demonstrating improved sample quality and competitiveness with higher-order methods through strategic evaluation placement.

Contribution

A new training-free first-order sampler with improved accuracy by strategic evaluation placement, provably approximating the ideal trajectory and achieving first-order convergence.

Findings

Consistently improves sample quality at the same NFE across benchmarks.

Can outperform some higher-order samplers in practice.

Highlights the importance of evaluation placement in diffusion sampling.

Abstract

Higher-order ODE solvers have become a standard tool for accelerating diffusion probabilistic model (DPM) sampling, motivating the widespread view that first-order methods are inherently slower and that increasing discretization order is the primary path to faster generation. This paper challenges this belief and revisits acceleration from a complementary angle: beyond solver order, the placement of DPM evaluations along the reverse-time dynamics can substantially affect sampling accuracy in the low-neural function evaluation (NFE) regime. We propose a novel training-free, first-order sampler whose leading discretization error has the opposite sign to that of DDIM. Algorithmically, the method approximates the forward-value evaluation via a cheap one-step lookahead predictor. We provide theoretical guarantees showing that the resulting sampler provably approximates the ideal forward-value trajectory while retaining first-order convergence. Empirically, across standard image generation benchmarks (CIFAR-10, ImageNet, FFHQ, and LSUN), the proposed sampler consistently improves sample quality under the same NFE budget and can be competitive with, and sometimes outperform, state-of-the-art higher-order samplers. Overall, the results suggest that the placement of DPM evaluations provides an additional and largely independent design angle for accelerating diffusion sampling.