Hallucination Detection with Small Language Models

TL;DR

This paper presents a scalable framework using multiple small language models to detect hallucinations in large language model responses by verifying answer consistency with retrieved context, improving detection accuracy.

Contribution

It introduces a novel approach combining small language models for hallucination detection, enhancing reliability without ground truth in question-answering tasks.

Findings

10% improvement in F1 scores for hallucination detection

Effective verification of partially correct responses

Scalable and efficient for practical applications

Abstract

Since the introduction of ChatGPT, large language models (LLMs) have demonstrated significant utility in various tasks, such as answering questions through retrieval-augmented generation. Context can be retrieved using a vectorized database, serving as a foundation for LLMs to generate responses. However, hallucinations in responses can undermine the reliability of LLMs in practical applications, and they are not easily detectable in the absence of ground truth, particularly in question-and-answer scenarios. This paper proposes a framework that integrates multiple small language models to verify responses generated by LLMs using the retrieved context from a vectorized database. By breaking down the responses into individual sentences and utilizing the probability of generating "Yes" tokens from the outputs of multiple models for a given set of questions, responses, and relevant context, hallucinations can be detected. The proposed framework is validated through experiments with real datasets comprising over 100 sets of questions, answers, and contexts, including responses with fully and partially correct sentences. The results demonstrate a 10\% improvement in F1 scores for detecting correct responses compared to hallucinations, indicating that multiple small language models can be effectively employed for answer verification, providing a scalable and efficient solution for both academic and practical applications.