The Illusion of Certainty: Uncertainty Quantification for LLMs Fails under Ambiguity

TL;DR

This paper reveals that existing uncertainty quantification methods for large language models perform poorly under ambiguous language, highlighting a critical shortcoming and the need for new approaches.

Contribution

The authors introduce the first ambiguous QA datasets with ground-truth answer distributions and demonstrate the limitations of current UQ methods under ambiguity.

Findings

Current UQ methods degrade to near-random performance on ambiguous data.

Performance deterioration is consistent across different estimation paradigms.

Theoretical analysis explains the fundamental limitations of existing estimators under ambiguity.

Abstract

Accurate uncertainty quantification (UQ) in Large Language Models (LLMs) is critical for trustworthy deployment. While real-world language is inherently ambiguous, reflecting aleatoric uncertainty, existing UQ methods are typically benchmarked against tasks with no ambiguity. In this work, we demonstrate that while current uncertainty estimators perform well under the restrictive assumption of no ambiguity, they degrade to close-to-random performance on ambiguous data. To this end, we introduce MAQA* and AmbigQA*, the first ambiguous question-answering (QA) datasets equipped with ground-truth answer distributions estimated from factual co-occurrence. We find this performance deterioration to be consistent across different estimation paradigms: using the predictive distribution itself, internal representations throughout the model, and an ensemble of models. We show that this phenomenon can be theoretically explained, revealing that predictive-distribution and ensemble-based estimators are fundamentally limited under ambiguity. Overall, our study reveals a key shortcoming of current UQ methods for LLMs and motivates a rethinking of current modeling paradigms.