Enhancing Hallucination Detection through Noise Injection

TL;DR

This paper introduces a simple, training-free method that enhances hallucination detection in large language models by injecting noise into model parameters, leveraging Bayesian uncertainty to improve accuracy across various datasets and architectures.

Contribution

The paper proposes a novel noise injection technique during sampling that significantly improves hallucination detection without additional training.

Findings

Improved hallucination detection accuracy across multiple datasets.

Enhanced detection performance by perturbing model parameters during sampling.

Method is effective across diverse model architectures and uncertainty metrics.

Abstract

Large Language Models (LLMs) are prone to generating plausible yet incorrect responses, known as hallucinations. Effectively detecting hallucinations is therefore crucial for the safe deployment of LLMs. Recent research has linked hallucinations to model uncertainty, suggesting that hallucinations can be detected by measuring dispersion over answer distributions obtained from multiple samples drawn from a model. While drawing from the distribution over tokens defined by the model is a natural way to obtain samples, in this work, we argue that it is suboptimal for the purpose of detecting hallucinations. We show that detection can be improved significantly by taking into account model uncertainty in the Bayesian sense. To this end, we propose a very simple, training-free approach based on perturbing an appropriate subset of model parameters, or equivalently hidden unit activations, during sampling. We demonstrate that our approach significantly improves inference-time hallucination detection over standard sampling across diverse datasets, model architectures, and uncertainty metrics.