Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspection

TL;DR

This paper investigates the internal signals of neural machine translation models that indicate hallucinations, and develops a lightweight detector that outperforms existing methods in identifying hallucinated outputs across multiple language pairs.

Contribution

It introduces a novel analysis of model internals to detect hallucinations and proposes an effective, lightweight hallucination detection method for neural machine translation.

Findings

Internal model symptoms reliably indicate hallucinations

The proposed detector outperforms baselines and large pre-trained models

Effective across multiple language pairs and datasets

Abstract

Neural sequence generation models are known to "hallucinate", by producing outputs that are unrelated to the source text. These hallucinations are potentially harmful, yet it remains unclear in what conditions they arise and how to mitigate their impact. In this work, we first identify internal model symptoms of hallucinations by analyzing the relative token contributions to the generation in contrastive hallucinated vs. non-hallucinated outputs generated via source perturbations. We then show that these symptoms are reliable indicators of natural hallucinations, by using them to design a lightweight hallucination detector which outperforms both model-free baselines and strong classifiers based on quality estimation or large pre-trained models on manually annotated English-Chinese and German-English translation test beds.