Label-Confidence-Aware Uncertainty Estimation in Natural Language Generation

TL;DR

This paper introduces a label-confidence-aware uncertainty estimation method for LLMs that accounts for biases from greedy decoding, improving the reliability of uncertainty quantification in NLP tasks.

Contribution

It proposes a novel LCA uncertainty estimation approach based on KL divergence that considers label source biases, enhancing model reliability.

Findings

Effective in capturing differences in sampling results and label sources.

Improves the stability and reliability of uncertainty estimates.

Demonstrates better performance across various LLMs and datasets.

Abstract

Large Language Models (LLMs) display formidable capabilities in generative tasks but also pose potential risks due to their tendency to generate hallucinatory responses. Uncertainty Quantification (UQ), the evaluation of model output reliability, is crucial for ensuring the safety and robustness of AI systems. Recent studies have concentrated on model uncertainty by analyzing the relationship between output entropy under various sampling conditions and the corresponding labels. However, these methods primarily focus on measuring model entropy with precision to capture response characteristics, often neglecting the uncertainties associated with greedy decoding results-the sources of model labels, which can lead to biased classification outcomes. In this paper, we explore the biases introduced by greedy decoding and propose a label-confidence-aware (LCA) uncertainty estimation based on Kullback-Leibler (KL) divergence bridging between samples and label source, thus enhancing the reliability and stability of uncertainty assessments. Our empirical evaluations across a range of popular LLMs and NLP datasets reveal that different label sources can indeed affect classification, and that our approach can effectively capture differences in sampling results and label sources, demonstrating more effective uncertainty estimation.