Rethinking LLM Uncertainty: A Multi-Agent Approach to Estimating Black-Box Model Uncertainty

TL;DR

This paper introduces a multi-agent method called DiverseAgentEntropy for more accurately estimating the uncertainty of black-box large language models, addressing limitations of existing self-consistency approaches.

Contribution

The paper proposes a novel, theoretically-grounded multi-agent approach that improves uncertainty estimation and hallucination detection in black-box LLMs.

Findings

Outperforms existing self-consistency methods in uncertainty estimation

More accurately detects model hallucinations

Addresses biases in parametric knowledge retrieval

Abstract

Quantifying uncertainty in black-box LLMs is vital for reliable responses and scalable oversight. Existing methods, which gauge a model's uncertainty through evaluating self-consistency in responses to the target query, can be misleading: an LLM may confidently provide an incorrect answer to a target query, yet give a confident and accurate answer to that same target query when answering a knowledge-preserving perturbation of the query. We systematically analyze the model behaviors and demonstrate that this discrepancy stems from suboptimal retrieval of parametric knowledge, often due to contextual biases that prevent consistent access to stored knowledge. We then introduce DiverseAgentEntropy, a novel, theoretically-grounded method employing multi-agent interaction across diverse query variations for uncertainty estimation of black-box LLMs. This approach more accurately assesses an LLM's true uncertainty and improves hallucination detection, outperforming existing self-consistency based techniques.