Uncertainty Quantification for Large Language Diffusion Models

TL;DR

This paper introduces a novel, efficient uncertainty quantification method for Large Language Diffusion Models, enabling reliable hallucination detection with minimal computational cost.

Contribution

It presents the first systematic UQ approach for LLDMs, leveraging iterative denoising signals and adapting a state-of-the-art method for improved efficiency.

Findings

Achieves near state-of-the-art uncertainty detection with up to 100x less computation.

Utilizes intermediate generations and denoising dynamics for zero-shot UQ signals.

Demonstrates effectiveness across multiple tasks, datasets, and models.

Abstract

Large Language Diffusion Models (LLDMs) are emerging as an alternative to autoregressive models, offering faster inference through higher parallelism. Similar to autoregressive LLMs, they remain prone to hallucinations, making reliable uncertainty quantification (UQ) crucial for safe deployment. However, existing UQ methods are fundamentally misaligned with this new paradigm: they assume autoregressive factorization or use expensive repeated sampling, negating the efficiency of LLDMs. In this work, we present the first systematic study of UQ for LLDMs and propose lightweight, zero-shot uncertainty signals derived from the iterative denoising process, leveraging intermediate generations, token remasking dynamics, and denoising complexity. We further adapt a state-of-the-art UQ method to LLDMs by combining masked diffusion likelihoods with trajectory-based semantic dissimilarity. We prove that expected trajectory dissimilarity lower bounds the masked diffusion training objective, which motivates its usage as an uncertainty score. Comprehensive experiments across three tasks, eight datasets, and two models show that our method achieves a great cost-performance trade-off: it approaches the strongest sampling-based baselines while incurring up to 100x lower computational overhead. Our work demonstrates that LLDMs can deliver both fast inference and reliable hallucination detection simultaneously.