CD4LM: Consistency Distillation and aDaptive Decoding for Diffusion Language Models

TL;DR

CD4LM introduces a novel framework combining consistency distillation and adaptive decoding to enable highly parallel, efficient diffusion language model generation without sacrificing quality, outperforming existing methods on benchmarks.

Contribution

The paper proposes CD4LM, a new approach that decouples training from inference in diffusion language models, enabling low-evaluation, high-quality, parallel decoding.

Findings

Matches LLaDA baseline with 5.18x speedup on GSM8K

Achieves 3.62x average speedup across benchmarks

Outperforms accuracy-efficiency Pareto frontier

Abstract

Autoregressive large language models achieve strong results on many benchmarks, but decoding remains fundamentally latency-limited by sequential dependence on previously generated tokens. Diffusion language models (DLMs) promise parallel generation but suffer from a fundamental static-to-dynamic misalignment: Training optimizes local transitions under fixed schedules, whereas efficient inference requires adaptive "long-jump" refinements through unseen states. Our goal is to enable highly parallel decoding for DLMs with low number of function evaluations while preserving generation quality. To achieve this, we propose CD4LM, a framework that decouples training from inference via Discrete-Space Consistency Distillation (DSCD) and Confidence-Adaptive Decoding (CAD). Unlike standard objectives, DSCD trains a student to be trajectory-invariant, mapping diverse noisy states directly to the clean distribution. This intrinsic robustness enables CAD to dynamically allocate compute resources based on token confidence, aggressively skipping steps without the quality collapse typical of heuristic acceleration. On GSM8K, CD4LM matches the LLaDA baseline with a 5.18x wall-clock speedup; across code and math benchmarks, it strictly dominates the accuracy-efficiency Pareto frontier, achieving a 3.62x mean speedup while improving average accuracy. Code is available at https://github.com/yihao-liang/CDLM