SLM Meets LLM: Balancing Latency, Interpretability and Consistency in Hallucination Detection

TL;DR

This paper introduces a hybrid framework combining a small language model and a large language model to enable real-time, interpretable hallucination detection in LLMs, balancing latency, interpretability, and consistency.

Contribution

It presents a novel two-stage approach that uses a small language model for quick detection and an LLM for detailed explanations, optimizing real-time hallucination detection.

Findings

Effective prompting techniques improve explanation alignment.

Framework reduces latency while maintaining interpretability.

Empirical results show enhanced detection accuracy and user experience.

Abstract

Large language models (LLMs) are highly capable but face latency challenges in real-time applications, such as conducting online hallucination detection. To overcome this issue, we propose a novel framework that leverages a small language model (SLM) classifier for initial detection, followed by a LLM as constrained reasoner to generate detailed explanations for detected hallucinated content. This study optimizes the real-time interpretable hallucination detection by introducing effective prompting techniques that align LLM-generated explanations with SLM decisions. Empirical experiment results demonstrate its effectiveness, thereby enhancing the overall user experience.