Towards a Systematic Evaluation of Hallucinations in Large-Vision Language Models

TL;DR

This paper introduces HALLUCINOGEN, a benchmark for evaluating hallucinations in large vision-language models through contextual reasoning prompts, revealing current models' vulnerability to hallucination attacks in complex tasks.

Contribution

The paper presents a novel benchmark that systematically assesses hallucination in LVLMs using contextual reasoning prompts and hallucination attacks, focusing on implicit reasoning about salient and latent entities.

Findings

Current LVLMs are susceptible to hallucination attacks.

Evaluation across multiple models shows varying robustness.

Hallucination mitigation strategies have limited effectiveness.

Abstract

Large Vision-Language Models (LVLMs) have demonstrated remarkable performance in complex multimodal tasks. However, these models still suffer from hallucinations, particularly when required to implicitly recognize or infer diverse visual entities from images for complex vision-language tasks. To address this challenge, we propose HALLUCINOGEN, a novel visual question answering (VQA) benchmark that employs contextual reasoning prompts as hallucination attacks to evaluate the extent of hallucination in state-of-the-art LVLMs. Our benchmark provides a comprehensive study of the implicit reasoning capabilities of these models by first categorizing visual entities based on the ease of recognition in an image as either salient (prominent, visibly recognizable objects such as a car) or latent entities (such as identifying a disease from a chest X-ray), which are not readily visible and require domain knowledge or contextual reasoning for accurate inference. Next, we design hallucination attacks for both types of entities to assess hallucinations in LVLMs while performing various vision-language tasks, such as locating or reasoning about specific entities within an image, where models must perform implicit reasoning by verifying the existence of the queried entity within the image before generating responses. Finally, our extensive evaluations of eleven LVLMs, including powerful open-source models (like LLaMA-3.2 and DeepSeek-V2), commercial models like Gemini, and two hallucination mitigation strategies across multiple datasets, demonstrate that current LVLMs remain susceptible to hallucination attacks.