The Map of Misbelief: Tracing Intrinsic and Extrinsic Hallucinations Through Attention Patterns

TL;DR

This paper introduces a new framework for distinguishing and detecting intrinsic and extrinsic hallucinations in large language models using attention patterns, improving interpretability and detection accuracy.

Contribution

It proposes a novel evaluation framework and attention aggregation methods that enhance hallucination detection and interpretability, especially for intrinsic hallucinations.

Findings

Sampling methods detect extrinsic hallucinations effectively.

Attention-based aggregation improves intrinsic hallucination detection.

Attention signals are valuable for quantifying model uncertainty.

Abstract

Large Language Models (LLMs) are increasingly deployed in safety-critical domains, yet remain susceptible to hallucinations. While prior works have proposed confidence representation methods for hallucination detection, most of these approaches rely on computationally expensive sampling strategies and often disregard the distinction between hallucination types. In this work, we introduce a principled evaluation framework that differentiates between extrinsic and intrinsic hallucination categories and evaluates detection performance across a suite of curated benchmarks. In addition, we leverage a recent attention-based uncertainty quantification algorithm and propose novel attention aggregation strategies that improve both interpretability and hallucination detection performance. Our experimental findings reveal that sampling-based methods like Semantic Entropy are effective for detecting extrinsic hallucinations but generally fail on intrinsic ones. In contrast, our method, which aggregates attention over input tokens, is better suited for intrinsic hallucinations. These insights provide new directions for aligning detection strategies with the nature of hallucination and highlight attention as a rich signal for quantifying model uncertainty.