What Color Is It? A Text-Interference Multimodal Hallucination Benchmark

TL;DR

This paper introduces a new benchmark dataset to evaluate and understand color hallucination issues in multimodal large models, aiming to improve their robustness against visual perception interference.

Contribution

The paper presents the 'What Color Is It' dataset for testing visual hallucinations in MLMs and explores causes and solutions for these hallucinations.

Findings

MLMs are susceptible to color hallucinations due to visual interference.

The dataset effectively triggers single-modality hallucinations.

Potential solutions can mitigate hallucination effects.

Abstract

With the rapid advancement of Large Models, numerous text-and-vision-fused Multimodal Large Models (MLMs) have emerged. However, these MLMs remain susceptible to informational interference in visual perception, particularly in color perception, which introduces an additional risk of hallucination. To validate this hypothesis, we introduce the "What Color Is It" dataset, a novel benchmark constructed using a simple method to trigger single-modality visual hallucination in MLMs. Based on this dataset, we further investigate the underlying causes of hallucination in the visual modality of MLMs and propose potential solutions to enhance their robustness.