LLM Internal States Reveal Hallucination Risk Faced With a Query
Ziwei Ji, Delong Chen, Etsuko Ishii, Samuel Cahyawijaya, Yejin Bang,, Bryan Wilie, Pascale Fung

TL;DR
This paper investigates whether large language models can internally assess their own hallucination risk by analyzing their internal states across diverse tasks, achieving over 84% accuracy in self-estimation.
Contribution
It demonstrates that LLM internal states encode information about training data exposure and hallucination likelihood, enabling self-assessment of uncertainty.
Findings
Internal states indicate training data familiarity.
Internal states correlate with hallucination risk.
Self-assessment accuracy reaches 84.32%.
Abstract
The hallucination problem of Large Language Models (LLMs) significantly limits their reliability and trustworthiness. Humans have a self-awareness process that allows us to recognize what we don't know when faced with queries. Inspired by this, our paper investigates whether LLMs can estimate their own hallucination risk before response generation. We analyze the internal mechanisms of LLMs broadly both in terms of training data sources and across 15 diverse Natural Language Generation (NLG) tasks, spanning over 700 datasets. Our empirical analysis reveals two key insights: (1) LLM internal states indicate whether they have seen the query in training data or not; and (2) LLM internal states show they are likely to hallucinate or not regarding the query. Our study explores particular neurons, activation layers, and tokens that play a crucial role in the LLM perception of uncertainty and…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsHallucinations in medical conditions
