Provenance: A Light-weight Fact-checker for Retrieval Augmented LLM Generation Output
Hithesh Sankararaman, Mohammed Nasheed Yasin, Tanner Sorensen, and Alessandro Di Bari, Andreas Stolcke

TL;DR
This paper introduces a lightweight, open-source fact-checking method for retrieval-augmented generation outputs that uses NLI models to detect nonfactual content efficiently without relying on large language models.
Contribution
The authors propose a novel, low-cost fact-checking approach using NLI models for RAG outputs, enabling effective detection and correction of hallucinations without LLM fine-tuning.
Findings
High AUC scores across multiple datasets
Low latency and cost at run-time
No need for LLM fine-tuning
Abstract
We present a light-weight approach for detecting nonfactual outputs from retrieval-augmented generation (RAG). Given a context and putative output, we compute a factuality score that can be thresholded to yield a binary decision to check the results of LLM-based question-answering, summarization, or other systems. Unlike factuality checkers that themselves rely on LLMs, we use compact, open-source natural language inference (NLI) models that yield a freely accessible solution with low latency and low cost at run-time, and no need for LLM fine-tuning. The approach also enables downstream mitigation and correction of hallucinations, by tracing them back to specific context chunks. Our experiments show high area under the ROC curve (AUC) across a wide range of relevant open source datasets, indicating the effectiveness of our method for fact-checking RAG output.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Taxonomy
TopicsScientific Computing and Data Management · Data Quality and Management · Semantic Web and Ontologies
MethodsRefunds@Expedia|||How do I get a full refund from Expedia? · Linear Layer · Softmax · Dropout · Dense Connections · Layer Normalization · Linear Warmup With Linear Decay · WordPiece · Adam · Attention Is All You Need
