Faithfulness-Aware Uncertainty Quantification for Fact-Checking the Output of Retrieval Augmented Generation

TL;DR

This paper introduces FRANQ, a novel method for detecting hallucinations in retrieval-augmented generation outputs by applying uncertainty quantification conditioned on faithfulness to retrieved evidence.

Contribution

FRANQ uniquely distinguishes factuality from faithfulness, improving hallucination detection in RAG outputs through specialized uncertainty quantification techniques.

Findings

FRANQ outperforms existing methods in factual error detection.

A new dataset for evaluating faithfulness and factuality in long-form QA.

Extensive experiments validate FRANQ's effectiveness across multiple datasets and models.

Abstract

Large Language Models (LLMs) enhanced with retrieval, an approach known as Retrieval-Augmented Generation (RAG), have achieved strong performance in open-domain question answering. However, RAG remains prone to hallucinations: factually incorrect outputs may arise from inaccuracies in the model's internal knowledge and the retrieved context. Existing approaches to mitigating hallucinations often conflate factuality with faithfulness to the retrieved evidence, incorrectly labeling factually correct statements as hallucinations if they are not explicitly supported by the retrieval. In this paper, we introduce FRANQ, a new method for hallucination detection in RAG outputs. FRANQ applies distinct uncertainty quantification (UQ) techniques to estimate factuality, conditioning on whether a statement is faithful to the retrieved context. To evaluate FRANQ and competing UQ methods, we construct a new long-form question answering dataset annotated for both factuality and faithfulness, combining automated labeling with manual validation of challenging cases. Extensive experiments across multiple datasets, tasks, and LLMs show that FRANQ achieves more accurate detection of factual errors in RAG-generated responses compared to existing approaches.