Rethinking the Uncertainty: A Critical Review and Analysis in the Era of Large Language Models

TL;DR

This paper critically reviews uncertainty estimation in Large Language Models, proposing a comprehensive framework to categorize and understand different uncertainty types to improve reliability and safety in AI applications.

Contribution

It introduces a novel framework for identifying and understanding uncertainties in LLMs, addressing limitations of current methods and guiding future research.

Findings

Current uncertainty estimation methods often misidentify sources of uncertainty

The framework categorizes uncertainties specific to LLMs

Insights into improving safety-critical applications

Abstract

In recent years, Large Language Models (LLMs) have become fundamental to a broad spectrum of artificial intelligence applications. As the use of LLMs expands, precisely estimating the uncertainty in their predictions has become crucial. Current methods often struggle to accurately identify, measure, and address the true uncertainty, with many focusing primarily on estimating model confidence. This discrepancy is largely due to an incomplete understanding of where, when, and how uncertainties are injected into models. This paper introduces a comprehensive framework specifically designed to identify and understand the types and sources of uncertainty, aligned with the unique characteristics of LLMs. Our framework enhances the understanding of the diverse landscape of uncertainties by systematically categorizing and defining each type, establishing a solid foundation for developing targeted methods that can precisely quantify these uncertainties. We also provide a detailed introduction to key related concepts and examine the limitations of current methods in mission-critical and safety-sensitive applications. The paper concludes with a perspective on future directions aimed at enhancing the reliability and practical adoption of these methods in real-world scenarios.