Sample Transform Cost-Based Training-Free Hallucination Detector for Large Language Models

TL;DR

This paper introduces a training-free hallucination detector for large language models that uses Wasserstein distances between token embedding samples to measure distribution complexity, effectively identifying hallucinations.

Contribution

It proposes a novel, training-free method based on optimal-transport distances to detect hallucinations in LLMs, applicable even in black-box settings.

Findings

AvgWD and EigenWD outperform some uncertainty baselines.

The methods are effective across different models and datasets.

Distribution complexity signals improve hallucination detection.

Abstract

Hallucinations in large language models (LLMs) remain a central obstacle to trustworthy deployment, motivating detectors that are accurate, lightweight, and broadly applicable. Since an LLM with a prompt defines a conditional distribution, we argue that the complexity of the distribution is an indicator of hallucination. However, the density of the distribution is unknown and the samples (i.e., responses generated for the prompt) are discrete distributions, which leads to a significant challenge in quantifying the complexity of the distribution. We propose to compute the optimal-transport distances between the sets of token embeddings of pairwise samples, which yields a Wasserstein distance matrix measuring the costs of transforming between the samples. This Wasserstein distance matrix provides a means to quantify the complexity of the distribution defined by the LLM with the prompt. Based on the Wasserstein distance matrix, we derive two complementary signals: AvgWD, measuring the average cost, and EigenWD, measuring the cost complexity. This leads to a training-free detector for hallucinations in LLMs. We further extend the framework to black-box LLMs via teacher forcing with an accessible teacher model. Experiments show that AvgWD and EigenWD are competitive with strong uncertainty baselines and provide complementary behavior across models and datasets, highlighting distribution complexity as an effective signal for LLM truthfulness.