Uncertainty Estimation and Quantification for LLMs: A Simple Supervised Approach

TL;DR

This paper introduces a supervised method for uncertainty estimation in large language models, leveraging hidden activations to improve calibration and transferability across tasks and model access levels.

Contribution

It proposes a novel supervised approach that utilizes hidden neuron activations for uncertainty estimation in LLMs, addressing a gap in existing calibration methods.

Findings

Enhanced uncertainty estimation using hidden activations.

Improved calibration performance with the proposed method.

Robust transferability in out-of-distribution scenarios.

Abstract

In this paper, we study the problem of uncertainty estimation and calibration for LLMs. We begin by formulating the uncertainty estimation problem, a relevant yet underexplored area in existing literature. We then propose a supervised approach that leverages labeled datasets to estimate the uncertainty in LLMs' responses. Based on the formulation, we illustrate the difference between the uncertainty estimation for LLMs and that for standard ML models and explain why the hidden neurons of the LLMs may contain uncertainty information. Our designed approach demonstrates the benefits of utilizing hidden activations to enhance uncertainty estimation across various tasks and shows robust transferability in out-of-distribution settings. We distinguish the uncertainty estimation task from the uncertainty calibration task and show that better uncertainty estimation leads to better calibration performance. Furthermore, our method is easy to implement and adaptable to different levels of model accessibility including black box, grey box, and white box.