VIHD: Visual Intervention-based Hallucination Detection for Medical Visual Question Answering

TL;DR

This paper introduces VIHD, a novel hallucination detection method for medical visual question answering that uses visual token masking and semantic entropy calibration to improve detection accuracy.

Contribution

VIHD leverages visual dependency probing and targeted visual intervention decoding to enhance hallucination detection in medical VQA models, outperforming existing methods.

Findings

VIHD outperforms state-of-the-art hallucination detection methods on three medical VQA benchmarks.

Visual dependency plays a crucial role in effective hallucination detection.

Calibrated semantic entropy (CSE) provides a reliable signal for hallucination identification.

Abstract

While medical Multimodal Large Language Models (MLLMs) have shown promise in assisting diagnosis, they still frequently generate hallucinated responses that appear linguistically plausible but lack visual evidence. Such hallucinations pose risks to clinical decision-making and necessitate effective detection. Existing introspective detection methods primarily perform uncertainty estimation or logical verification by analyzing model responses conditioned on original or perturbed inputs. However, such external perturbations are often heuristic and context-agnostic, which overlooks the internal cross-modal dependency between generated tokens and related visual tokens during decoding. To address this issue, we propose VIHD, a Visual Intervention-based Hallucination Detection method that leverages targeted visual token masking to calibrate semantic entropy for more effective hallucination detection. VIHD locates visually dominant decoder layers via Visual Dependency Probing (VDP), executes Visual Intervention Decoding (VID) via token masking to calibrate the semantic distribution, and quantifies the resulting Calibrated Semantic Entropy (CSE) as a reliable hallucination signal. Extensive experiments on three medical VQA benchmarks with two medical MLLMs demonstrate that VIHD consistently outperforms state-of-the-art methods, underscoring the importance of fine-grained visual dependency for hallucination detection. The code will be available at https://github.com/Jiayi-Chen-AU/VIHD