Accelerating Diffusion Large Language Models with SlowFast Sampling: The Three Golden Principles

TL;DR

This paper introduces SlowFast Sampling, a dynamic strategy for diffusion-based language models that adaptively balances exploration and speed, significantly improving inference efficiency while maintaining accuracy.

Contribution

The paper proposes a novel adaptive sampling method guided by three principles, enhancing diffusion LLMs' speed and flexibility over existing static strategies.

Findings

Achieves up to 15.63× speedup on LLaDA with minimal accuracy loss.

Reaches up to 34.22× speedup when combined with caching.

Outperforms autoregressive baselines like LLaMA3 8B in throughput.

Abstract

Diffusion-based language models (dLLMs) have emerged as a promising alternative to traditional autoregressive LLMs by enabling parallel token generation and significantly reducing inference latency. However, existing sampling strategies for dLLMs, such as confidence-based or semi-autoregressive decoding, often suffer from static behavior, leading to suboptimal efficiency and limited flexibility. In this paper, we propose SlowFast Sampling, a novel dynamic sampling strategy that adaptively alternates between exploratory and accelerated decoding stages. Our method is guided by three golden principles: certainty principle, convergence principle, and positional principle, which govern when and where tokens can be confidently and efficiently decoded. We further integrate our strategy with dLLM-Cache to reduce redundant computation. Extensive experiments across benchmarks and models show that SlowFast Sampling achieves up to 15.63$\times$ speedup on LLaDA with minimal accuracy drop, and up to 34.22$\times$ when combined with caching. Notably, our approach outperforms strong autoregressive baselines like LLaMA3 8B in throughput, demonstrating that well-designed sampling can unlock the full potential of dLLMs for fast and high-quality generation.