HAD: HAllucination Detection Language Models Based on a Comprehensive Hallucination Taxonomy

TL;DR

This paper introduces a comprehensive hallucination taxonomy and the HAD models, which detect, identify, and correct hallucinations in language models across multiple tasks, demonstrating state-of-the-art performance and robustness.

Contribution

It proposes a detailed hallucination taxonomy and a versatile HAD model that integrates detection, span identification, and correction in a single inference process.

Findings

HAD models outperform existing baselines on multiple benchmarks.

The models achieve state-of-the-art results on HaluEval, FactCHD, and FaithBench.

The approach is effective across various NLG tasks and out-of-domain data.

Abstract

The increasing reliance on natural language generation (NLG) models, particularly large language models, has raised concerns about the reliability and accuracy of their outputs. A key challenge is hallucination, where models produce plausible but incorrect information. As a result, hallucination detection has become a critical task. In this work, we introduce a comprehensive hallucination taxonomy with 11 categories across various NLG tasks and propose the HAllucination Detection (HAD) models https://github.com/pku0xff/HAD, which integrate hallucination detection, span-level identification, and correction into a single inference process. Trained on an elaborate synthetic dataset of about 90K samples, our HAD models are versatile and can be applied to various NLG tasks. We also carefully annotate a test set for hallucination detection, called HADTest, which contains 2,248 samples. Evaluations on in-domain and out-of-domain test sets show that our HAD models generally outperform the existing baselines, achieving state-of-the-art results on HaluEval, FactCHD, and FaithBench, confirming their robustness and versatility.