Osiris: A Lightweight Open-Source Hallucination Detection System

TL;DR

Osiris is a lightweight, open-source system for detecting hallucinations in retrieval-augmented generation models, offering improved recall with fewer parameters compared to larger models like GPT-4.

Contribution

We introduce a perturbed multi-hop QA dataset with hallucinations and demonstrate that fine-tuning a 7B model on this dataset improves hallucination detection performance.

Findings

Better recall than GPT-4 on RAGTruth benchmark

Competitive precision and accuracy with fewer parameters

Open-source code available for community use

Abstract

Retrieval-Augmented Generation (RAG) systems have gained widespread adoption by application builders because they leverage sources of truth to enable Large Language Models (LLMs) to generate more factually sound responses. However, hallucinations, instances of LLM responses that are unfaithful to the provided context, often prevent these systems from being deployed in production environments. Current hallucination detection methods typically involve human evaluation or the use of closed-source models to review RAG system outputs for hallucinations. Both human evaluators and closed-source models suffer from scaling issues due to their high costs and slow inference speeds. In this work, we introduce a perturbed multi-hop QA dataset with induced hallucinations. Via supervised fine-tuning on our dataset, we achieve better recall with a 7B model than GPT-4o on the RAGTruth hallucination detection benchmark and offer competitive performance on precision and accuracy, all while using a fraction of the parameters. Code is released at our repository.