Seeing is Believing: Rich-Context Hallucination Detection for MLLMs via Backward Visual Grounding

TL;DR

This paper introduces VBackChecker, a novel reference-free framework for detecting hallucinations in multimodal large language models by verifying visual-text consistency, and establishes a new benchmark R^2-HalBench with high-quality annotations.

Contribution

The paper presents VBackChecker, a new interpretability-driven, reference-free hallucination detection method for MLLMs, and introduces R^2-HalBench, a comprehensive benchmark with rich-context annotations.

Findings

VBackChecker outperforms prior methods in hallucination detection accuracy.

Achieves over 10% improvement in pixel-level grounding tasks.

R^2-HalBench provides diverse, high-quality annotations for real-world scenarios.

Abstract

Multimodal Large Language Models (MLLMs) have unlocked powerful cross-modal capabilities, but still significantly suffer from hallucinations. As such, accurate detection of hallucinations in MLLMs is imperative for ensuring their reliability in practical applications. To this end, guided by the principle of "Seeing is Believing", we introduce VBackChecker, a novel reference-free hallucination detection framework that verifies the consistency of MLLMgenerated responses with visual inputs, by leveraging a pixellevel Grounding LLM equipped with reasoning and referring segmentation capabilities. This reference-free framework not only effectively handles rich-context scenarios, but also offers interpretability. To facilitate this, an innovative pipeline is accordingly designed for generating instruction-tuning data (R-Instruct), featuring rich-context descriptions, grounding masks, and hard negative samples. We further establish R^2 -HalBench, a new hallucination benchmark for MLLMs, which, unlike previous benchmarks, encompasses real-world, rich-context descriptions from 18 MLLMs with high-quality annotations, spanning diverse object-, attribute, and relationship-level details. VBackChecker outperforms prior complex frameworks and achieves state-of-the-art performance on R^2 -HalBench, even rivaling GPT-4o's capabilities in hallucination detection. It also surpasses prior methods in the pixel-level grounding task, achieving over a 10% improvement. All codes, data, and models are available at https://github.com/PinxueGuo/VBackChecker.