ZEUS: Accelerating Diffusion Models with Only Second-Order Predictor

TL;DR

ZEUS is a training-free acceleration method for diffusion models that uses second-order prediction and skipping strategies to significantly reduce inference time while preserving quality.

Contribution

It introduces a novel second-order predictor and skipping scheme that enhance inference speed without architectural changes or feature caching.

Findings

Achieves up to 3.2x speedup in image and video generation.

Consistently outperforms recent training-free acceleration baselines.

Maintains perceptual quality despite aggressive speedups.

Abstract

Denoising generative models deliver high-fidelity generation but remain bottlenecked by inference latency due to the many iterative denoiser calls required during sampling. Training-free acceleration methods reduce latency by either sparsifying the model architecture or shortening the sampling trajectory. Current training-free acceleration methods are more complex than necessary: higher-order predictors amplify error under aggressive speedups, and architectural modifications hinder deployment. Beyond 2x acceleration, step skipping creates structural scarcity -- at most one fresh evaluation per local window -- leaving the computed output and its backward difference as the only causally grounded information. Based on this, we propose ZEUS, an acceleration method that predicts reduced denoiser evaluations using a second-order predictor, and stabilizes aggressive consecutive skipping with an interleaved scheme that avoids back-to-back extrapolations. ZEUS adds essentially zero overhead, no feature caches, and no architectural modifications, and it is compatible with different backbones, prediction objectives, and solver choices. Across image and video generation, ZEUS consistently improves the speed-fidelity performance over recent training-free baselines, achieving up to 3.2x end-to-end speedup while maintaining perceptual quality. Our code is available at: https://github.com/Ting-Justin-Jiang/ZEUS.