Tortoise and Hare Guidance: Accelerating Diffusion Model Inference with Multirate Integration

TL;DR

This paper introduces Tortoise and Hare Guidance (THG), a training-free method that accelerates diffusion model inference by using multirate integration, reducing computation by up to 30% while maintaining high image quality.

Contribution

The paper presents a novel multirate formulation for diffusion guidance that significantly reduces computation without retraining, outperforming existing accelerators.

Findings

Reduces function evaluations by up to 30%

Maintains high-fidelity image generation ($b4$ImageReward a0c= 0.032)

Outperforms state-of-the-art CFG accelerators under same computation budgets

Abstract

In this paper, we propose Tortoise and Hare Guidance (THG), a training-free strategy that accelerates diffusion sampling while maintaining high-fidelity generation. We demonstrate that the noise estimate and the additional guidance term exhibit markedly different sensitivity to numerical error by reformulating the classifier-free guidance (CFG) ODE as a multirate system of ODEs. Our error-bound analysis shows that the additional guidance branch is more robust to approximation, revealing substantial redundancy that conventional solvers fail to exploit. Building on this insight, THG significantly reduces the computation of the additional guidance: the noise estimate is integrated with the tortoise equation on the original, fine-grained timestep grid, while the additional guidance is integrated with the hare equation only on a coarse grid. We also introduce (i) an error-bound-aware timestep sampler that adaptively selects step sizes and (ii) a guidance-scale scheduler that stabilizes large extrapolation spans. THG reduces the number of function evaluations (NFE) by up to 30% with virtually no loss in generation fidelity ($\Delta$ImageReward $\leq$ 0.032) and outperforms state-of-the-art CFG-based training-free accelerators under identical computation budgets. Our findings highlight the potential of multirate formulations for diffusion solvers, paving the way for real-time high-quality image synthesis without any model retraining. The source code is available at https://github.com/yhlee-add/THG.