VOPE: Revisiting Hallucination of Vision-Language Models in Voluntary Imagination Task

TL;DR

This paper introduces VOPE, a new evaluation method for assessing hallucinations in vision-language models during voluntary imagination tasks, revealing high hallucination rates and limited mitigation effectiveness.

Contribution

The paper proposes VOPE, a novel presence evaluation method specifically designed for voluntary imagination tasks in LVLMs, highlighting the need for new hallucination mitigation strategies.

Findings

Most LVLMs hallucinate heavily during voluntary imagination.

Performance in presence evaluation is poor on imagined objects.

Existing mitigation methods have limited effect in these tasks.

Abstract

Most research on hallucinations in Large Vision-Language Models (LVLMs) focuses on factual description tasks that prohibit any output absent from the image. However, little attention has been paid to hallucinations in voluntary imagination tasks, e.g., story writing, where the models are expected to generate novel content beyond the given image. In these tasks, it is inappropriate to simply regard such imagined novel content as hallucinations. To address this limitation, we introduce Voluntary-imagined Object Presence Evaluation (VOPE)-a novel method to assess LVLMs' hallucinations in voluntary imagination tasks via presence evaluation. Specifically, VOPE poses recheck-based questions to evaluate how an LVLM interprets the presence of the imagined objects in its own response. The consistency between the model's interpretation and the object's presence in the image is then used to determine whether the model hallucinates when generating the response. We apply VOPE to several mainstream LVLMs and hallucination mitigation methods, revealing two key findings: (1) most LVLMs hallucinate heavily during voluntary imagination, and their performance in presence evaluation is notably poor on imagined objects; (2) existing hallucination mitigation methods show limited effect in voluntary imagination tasks, making this an important direction for future research.