SeSE: Black-Box Uncertainty Quantification for Large Language Models Based on Structural Information Theory

TL;DR

SeSE introduces a novel black-box uncertainty quantification framework for large language models that leverages structural information theory to provide more accurate and interpretable uncertainty estimates, especially for long-form outputs.

Contribution

SeSE is the first method to incorporate structural entropy for semantic uncertainty quantification in LLMs, applicable to both open- and closed-source models, and extends to granular long-form output analysis.

Findings

Outperforms strong baselines across 24 model-dataset pairs

Provides interpretable, granular uncertainty estimates for long-form outputs

Theoretically generalizes semantic entropy for LLMs.

Abstract

Reliable uncertainty quantification (UQ) is essential for deploying large language models (LLMs) in safety-critical scenarios, as it enables them to abstain from responding when uncertain, thereby avoiding hallucinations, i.e., plausible yet factually incorrect responses. However, while semantic UQ methods have achieved advanced performance, they overlook latent semantic structural information that could enable more precise uncertainty estimates. In this paper, we propose \underline{Se}mantic \underline{S}tructural \underline{E}ntropy ({SeSE}), a principled black-box UQ framework applicable to both open- and closed-source LLMs. To reveal the intrinsic structure of the semantic space, SeSE constructs its optimal hierarchical abstraction through an encoding tree with minimal structural entropy. The structural entropy of this encoding tree thus quantifies the inherent uncertainty within LLM semantic space after optimal compression. Additionally, unlike existing methods that primarily focus on simple short-form generation, we extent SeSE to provide interpretable, granular uncertainty estimation for long-form outputs. We theoretically prove that SeSE generalizes semantic entropy, the gold standard for UQ in LLMs, and empirically demonstrate its superior performance over strong baselines across 24 model-dataset combinations.