From Detection to Diagnosis: Advancing Hallucination Analysis with Automated Data Synthesis

TL;DR

This paper introduces a new paradigm for analyzing hallucinations in large language models by developing a diagnosis framework and automated data synthesis pipeline, leading to improved detection, explanation, and correction of hallucinations.

Contribution

It proposes the Hallucination Diagnosis Task, a novel approach that extends beyond detection to diagnosis, and develops HDG and HDM-4B-RL for systematic hallucination analysis and correction.

Findings

HDM-4B-RL surpasses previous detection models on HaluEval

The diagnosis model matches larger models in diagnostic tasks

Automated data synthesis enhances hallucination analysis capabilities

Abstract

Hallucinations in Large Language Models (LLMs), defined as the generation of content inconsistent with facts or context, represent a core obstacle to their reliable deployment in critical domains. Current research primarily focuses on binary "detection" approaches that, while capable of identifying hallucinations, fail to provide interpretable and actionable feedback for model improvement, thus limiting practical utility. To address this limitation, a new research paradigm is proposed, shifting from "detection" to "diagnosis". The Hallucination Diagnosis Task is introduced, a task which requires models to not only detect hallucinations, but also perform error localization, causal explanation, and content correction. We develop the Hallucination Diagnosis Generator (HDG), an automated pipeline that systematically generates high-quality training samples with rich diagnostic metadata from raw corpora through multi-dimensional augmentation strategies including controlled fact fabrication and reasoning chain perturbation. Using HDG-generated data, we train HDM-4B-RL, a 4-billion-parameter hallucination diagnosis model, employing Group Relative Policy Optimization (GRPO) with a comprehensive reward function incorporating structural, accuracy, and localization signals. Experimental results demonstrate that our model surpasses previous state-of-the-art detection models on the HaluEval benchmark while achieving comparable performance to advanced general-purpose models. In comprehensive diagnosis tasks, HDM-4B-RL matches the capabilities of larger general models while maintaining a smaller size. This work validates the feasibility and value of hallucination diagnosis, providing an effective methodology for building more trustworthy and reliable generative AI systems.