STDec: Spatio-Temporal Stability Guided Decoding for dLLMs

TL;DR

STDec introduces a spatio-temporal stability guided decoding method for diffusion large language models, enhancing decoding efficiency by leveraging local context and temporal consistency without retraining.

Contribution

It proposes a training-free, stability-based decoding approach that improves speed while maintaining performance across various benchmarks.

Findings

Achieves up to 14.17x speedup on MBPP with LLaDA.

Maintains comparable task performance scores.

Compatible with cache-based acceleration methods.

Abstract

Diffusion Large Language Models (dLLMs) have achieved rapid progress, viewed as a promising alternative to the autoregressive paradigm. However, most dLLM decoders still adopt a global confidence threshold, and do not explicitly model local context from neighboring decoded states or temporal consistency of predicted token IDs across steps. To address this issue, we propose a simple spatio-temporal stability guided decoding approach, named STDec. We observe strong spatio-temporal stability in dLLM decoding: newly decoded tokens tend to lie near decoded neighbors, and their predicted IDs often remain consistent across several denoising steps. Inspired by this stability, our STDec includes spatial-aware decoding and temporal-aware decoding. The spatial-aware decoding dynamically generates the token-adaptive threshold by aggregating the decoded states of nearby tokens. The temporal-aware decoding relaxes the decoding thresholds for tokens whose predicted token IDs remain consistent over denoising steps. Our STDec is training-free and remains compatible with cache-based acceleration methods. Across textual reasoning and multimodal understanding benchmarks, STDec substantially improves throughput while maintaining comparable task performance score. Notably, on MBPP with LLaDA, STDec achieves up to 14.17x speedup with a comparable score. Homepage: https://yzchen02.github.io/STDec.