Inv-Entropy: A Fully Probabilistic Framework for Uncertainty Quantification in Language Models

TL;DR

This paper introduces Inv-Entropy, a probabilistic framework for quantifying uncertainty in large language models, utilizing perturbations and a dual Markov chain perspective to improve reliability and evaluation of model uncertainty.

Contribution

It presents a novel fully probabilistic uncertainty measure, Inv-Entropy, along with a flexible framework and a genetic algorithm-based perturbation method for better uncertainty quantification in LLMs.

Findings

Inv-Entropy outperforms existing semantic UQ methods.

The framework supports various uncertainty measures and perturbation strategies.

The TSU metric effectively evaluates uncertainty without relying on correctness.

Abstract

Large language models (LLMs) have transformed natural language processing, but their reliable deployment requires effective uncertainty quantification (UQ). Existing UQ methods are often heuristic and lack a probabilistic interpretation. This paper begins by providing a theoretical justification for the role of perturbations in UQ for LLMs. We then introduce a dual random walk perspective, modeling input-output pairs as two Markov chains with transition probabilities defined by semantic similarity. Building on this, we propose a fully probabilistic framework based on an inverse model, which quantifies uncertainty by evaluating the diversity of the input space conditioned on a given output through systematic perturbations. Within this framework, we define a new uncertainty measure, Inv-Entropy. A key strength of our framework is its flexibility: it supports various definitions of uncertainty measures, embeddings, perturbation strategies, and similarity metrics. We also propose GAAP, a perturbation algorithm based on genetic algorithms, which enhances the diversity of sampled inputs. In addition, we introduce a new evaluation metric, Temperature Sensitivity of Uncertainty (TSU), which directly assesses uncertainty without relying on correctness as a proxy. Extensive experiments demonstrate that Inv-Entropy outperforms existing semantic UQ methods. The code to reproduce the results can be found at https://github.com/UMDataScienceLab/Uncertainty-Quantification-for-LLMs.