Reconsidering LLM Uncertainty Estimation Methods in the Wild

TL;DR

This paper systematically evaluates large language model uncertainty estimation methods in real-world scenarios, highlighting their sensitivities, robustness issues, and potential strategies for improvement in practical deployments.

Contribution

It provides a comprehensive assessment of existing UE methods under realistic conditions, revealing their vulnerabilities and proposing ensemble strategies for better performance.

Findings

Most UE methods are sensitive to threshold selection under distribution shifts.

UE methods are robust to chat history and typos but vulnerable to adversarial prompts.

Ensembling multiple UE scores improves performance significantly.

Abstract

Large Language Model (LLM) Uncertainty Estimation (UE) methods have become a crucial tool for detecting hallucinations in recent years. While numerous UE methods have been proposed, most existing studies evaluate them in isolated short-form QA settings using threshold-independent metrics such as AUROC or PRR. However, real-world deployment of UE methods introduces several challenges. In this work, we systematically examine four key aspects of deploying UE methods in practical settings. Specifically, we assess (1) the sensitivity of UE methods to decision threshold selection, (2) their robustness to query transformations such as typos, adversarial prompts, and prior chat history, (3) their applicability to long-form generation, and (4) strategies for handling multiple UE scores for a single query. Our evaluations on 19 UE methods reveal that most of them are highly sensitive to threshold selection when there is a distribution shift in the calibration dataset. While these methods generally exhibit robustness against previous chat history and typos, they are significantly vulnerable to adversarial prompts. Additionally, while existing UE methods can be adapted for long-form generation through various strategies, there remains considerable room for improvement. Lastly, ensembling multiple UE scores at test time provides a notable performance boost, which highlights its potential as a practical improvement strategy. Code is available at: https://github.com/duygunuryldz/uncertainty_in_the_wild.