Detecting Hallucinations in SpeechLLMs at Inference Time Using Attention Maps

TL;DR

This paper introduces attention-based metrics and classifiers to detect hallucinations in SpeechLLMs during inference, improving accuracy and generalization over existing methods.

Contribution

It proposes novel attention-derived metrics and lightweight classifiers for efficient hallucination detection in SpeechLLMs at inference time.

Findings

Outperforms uncertainty-based baselines with up to +0.23 PR-AUC improvement.

Achieves strong out-of-domain generalization in speech tasks.

Effective detection with approximately 100 attention heads.

Abstract

Hallucinations in Speech Large Language Models (SpeechLLMs) pose significant risks, yet existing detection methods typically rely on gold-standard outputs that are costly or impractical to obtain. Moreover, hallucination detection methods developed for text-based LLMs do not directly capture audio-specific signals. We investigate four attention-derived metrics: AUDIORATIO, AUDIOCONSISTENCY, AUDIOENTROPY, and TEXTENTROPY, designed to capture pathological attention patterns associated with hallucination, and train lightweight logistic regression classifiers on these features for efficient inference-time detection. Across automatic speech recognition and speech-to-text translation tasks, evaluations on Qwen-2-Audio and Voxtral-3B show that our approach outperforms uncertainty-based and prior attention-based baselines on in-domain data, achieving improvements of up to +0.23 PR-AUC, and generalises to out-of-domain ASR settings. We further find that strong performance can be achieved with approximately 100 attention heads, improving out-of-domain generalisation compared to using all heads. While effectiveness is model-dependent and task-specific training is required, our results demonstrate that attention patterns provide a valuable tool for hallucination detection in SpeechLLMs.