HALoGEN: Fantastic LLM Hallucinations and Where to Find Them

TL;DR

This paper introduces HALoGEN, a comprehensive benchmark with automatic verifiers to measure hallucinations in large language models across multiple domains, revealing high hallucination rates even in top models.

Contribution

The work provides a new benchmark and automatic verification framework for quantifying and analyzing hallucinations in LLMs, enabling more trustworthy model development.

Findings

Up to 86% hallucination rate in some models

Automatic verifiers effectively decompose and verify model outputs

Hallucinations stem from incorrect recollection, knowledge, or fabrication

Abstract

Despite their impressive ability to generate high-quality and fluent text, generative large language models (LLMs) also produce hallucinations: statements that are misaligned with established world knowledge or provided input context. However, measuring hallucination can be challenging, as having humans verify model generations on-the-fly is both expensive and time-consuming. In this work, we release HALoGEN, a comprehensive hallucination benchmark consisting of: (1) 10,923 prompts for generative models spanning nine domains including programming, scientific attribution, and summarization, and (2) automatic high-precision verifiers for each use case that decompose LLM generations into atomic units, and verify each unit against a high-quality knowledge source. We use this framework to evaluate ~150,000 generations from 14 language models, finding that even the best-performing models are riddled with hallucinations (sometimes up to 86% of generated atomic facts depending on the domain). We further define a novel error classification for LLM hallucinations based on whether they likely stem from incorrect recollection of training data (Type A errors), or incorrect knowledge in training data (Type B errors), or are fabrication (Type C errors). We hope our framework provides a foundation to enable the principled study of why generative models hallucinate, and advances the development of trustworthy large language models.