Hallucination Detection in LLMs: Fast and Memory-Efficient Fine-Tuned Models

TL;DR

This paper introduces a fast, memory-efficient method for training LLM ensembles that effectively detect hallucinations, making uncertainty estimation feasible in high-risk applications with minimal computational resources.

Contribution

The authors propose a novel approach enabling quick, memory-friendly ensemble training for LLMs to detect hallucinations, reducing computational costs significantly.

Findings

Ensembles can detect hallucinations effectively.

Only one GPU needed for training and inference.

Method is practical for high-risk AI applications.

Abstract

Uncertainty estimation is a necessary component when implementing AI in high-risk settings, such as autonomous cars, medicine, or insurances. Large Language Models (LLMs) have seen a surge in popularity in recent years, but they are subject to hallucinations, which may cause serious harm in high-risk settings. Despite their success, LLMs are expensive to train and run: they need a large amount of computations and memory, preventing the use of ensembling methods in practice. In this work, we present a novel method that allows for fast and memory-friendly training of LLM ensembles. We show that the resulting ensembles can detect hallucinations and are a viable approach in practice as only one GPU is needed for training and inference.