JointCQ: Improving Factual Hallucination Detection with Joint Claim and Query Generation

TL;DR

JointCQ is a novel framework that enhances factual hallucination detection in large language models by jointly generating claims and queries, leading to improved accuracy and reliability in identifying hallucinated content.

Contribution

The paper introduces a joint claim and query generation framework with a filtering strategy, significantly improving hallucination detection performance over existing methods.

Findings

Outperforms previous methods on multiple benchmarks

Enhances the reliability of hallucination detection

Provides more trustworthy language model outputs

Abstract

Current large language models (LLMs) often suffer from hallucination issues, i,e, generating content that appears factual but is actually unreliable. A typical hallucination detection pipeline involves response decomposition (i.e., claim extraction), query generation, evidence collection (i.e., search or retrieval), and claim verification. However, existing methods exhibit limitations in the first two stages, such as context loss during claim extraction and low specificity in query generation, resulting in degraded performance across the hallucination detection pipeline. In this work, we introduce JointCQ https://github.com/pku0xff/JointCQ, a joint claim-and-query generation framework designed to construct an effective and efficient claim-query generator. Our framework leverages elaborately designed evaluation criteria to filter synthesized training data, and finetunes a language model for joint claim extraction and query generation, providing reliable and informative inputs for downstream search and verification. Experimental results demonstrate that our method outperforms previous methods on multiple open-domain QA hallucination detection benchmarks, advancing the goal of more trustworthy and transparent language model systems.