Osiris: A Lightweight Open-Source Hallucination Detection System
Alex Shan, John Bauer, Christopher D. Manning

TL;DR
Osiris is a lightweight, open-source system for detecting hallucinations in retrieval-augmented generation models, offering improved recall with fewer parameters compared to larger models like GPT-4.
Contribution
We introduce a perturbed multi-hop QA dataset with hallucinations and demonstrate that fine-tuning a 7B model on this dataset improves hallucination detection performance.
Findings
Better recall than GPT-4 on RAGTruth benchmark
Competitive precision and accuracy with fewer parameters
Open-source code available for community use
Abstract
Retrieval-Augmented Generation (RAG) systems have gained widespread adoption by application builders because they leverage sources of truth to enable Large Language Models (LLMs) to generate more factually sound responses. However, hallucinations, instances of LLM responses that are unfaithful to the provided context, often prevent these systems from being deployed in production environments. Current hallucination detection methods typically involve human evaluation or the use of closed-source models to review RAG system outputs for hallucinations. Both human evaluators and closed-source models suffer from scaling issues due to their high costs and slow inference speeds. In this work, we introduce a perturbed multi-hop QA dataset with induced hallucinations. Via supervised fine-tuning on our dataset, we achieve better recall with a 7B model than GPT-4o on the RAGTruth hallucination…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMusic and Audio Processing · Adversarial Robustness in Machine Learning · Topic Modeling
MethodsRefunds@Expedia|||How do I get a full refund from Expedia? · Attention Is All You Need · Linear Warmup With Linear Decay · Dropout · Layer Normalization · Byte Pair Encoding · Attention Dropout · Softmax · Residual Connection · WordPiece
