Streaming-dLLM: Accelerating Diffusion LLMs via Suffix Pruning and Dynamic Decoding

TL;DR

Streaming-dLLM is a training-free framework that significantly accelerates diffusion large language models by pruning redundant suffix tokens and dynamically skipping unnecessary decoding steps, achieving up to 68.2X speedup without quality loss.

Contribution

It introduces a novel suffix pruning and dynamic decoding strategy for diffusion LLMs, addressing spatial and temporal inefficiencies without additional training.

Findings

Achieves up to 68.2X speedup in inference.

Maintains generation quality comparable to baseline.

Effective in reducing redundant computations during decoding.

Abstract

Diffusion Large Language Models (dLLMs) offer a compelling paradigm for natural language generation, leveraging parallel decoding and bidirectional attention to achieve superior global coherence compared to autoregressive models. While recent works have accelerated inference via KV cache reuse or heuristic decoding, they overlook the intrinsic inefficiencies within the block-wise diffusion process. Specifically, they suffer from spatial redundancy by modeling informative-sparse suffix regions uniformly and temporal inefficiency by applying fixed denoising schedules across all the decoding process. To address this, we propose Streaming-dLLM, a training-free framework that streamlines inference across both spatial and temporal dimensions. Spatially, we introduce attenuation guided suffix modeling to approximate the full context by pruning redundant mask tokens. Temporally, we employ a dynamic confidence aware strategy with an early exit mechanism, allowing the model to skip unnecessary iterations for converged tokens. Extensive experiments show that Streaming-dLLM achieves up to 68.2X speedup while maintaining generation quality, highlighting its effectiveness in diffusion decoding. The code is available at https://github.com/xiaoshideta/Streaming-dLLM.