Provenance: A Light-weight Fact-checker for Retrieval Augmented LLM Generation Output

TL;DR

This paper introduces a lightweight, open-source fact-checking method for retrieval-augmented generation outputs that uses NLI models to detect nonfactual content efficiently without relying on large language models.

Contribution

The authors propose a novel, low-cost fact-checking approach using NLI models for RAG outputs, enabling effective detection and correction of hallucinations without LLM fine-tuning.

Findings

High AUC scores across multiple datasets

Low latency and cost at run-time

No need for LLM fine-tuning

Abstract

We present a light-weight approach for detecting nonfactual outputs from retrieval-augmented generation (RAG). Given a context and putative output, we compute a factuality score that can be thresholded to yield a binary decision to check the results of LLM-based question-answering, summarization, or other systems. Unlike factuality checkers that themselves rely on LLMs, we use compact, open-source natural language inference (NLI) models that yield a freely accessible solution with low latency and low cost at run-time, and no need for LLM fine-tuning. The approach also enables downstream mitigation and correction of hallucinations, by tracing them back to specific context chunks. Our experiments show high area under the ROC curve (AUC) across a wide range of relevant open source datasets, indicating the effectiveness of our method for fact-checking RAG output.