Insight Over Sight: Exploring the Vision-Knowledge Conflicts in Multimodal LLMs

TL;DR

This paper investigates conflicts between visual data and internal knowledge in multimodal large language models, introducing a benchmark and framework to evaluate and improve their conflict-resolution abilities.

Contribution

It presents a novel automated framework and diagnostic benchmark for assessing vision-knowledge conflicts in MLLMs, along with analysis of existing mitigation strategies.

Findings

Approximately 20% of queries show over-reliance on parametric knowledge.

Existing mitigation strategies only partially reduce conflicts.

The proposed framework can scale to generate more conflict scenarios.

Abstract

This paper explores the problem of commonsense level vision-knowledge conflict in Multimodal Large Language Models (MLLMs), where visual information contradicts model's internal commonsense knowledge. To study this issue, we introduce an automated framework, augmented with human-in-the-loop quality control, to generate inputs designed to simulate and evaluate these conflicts in MLLMs. Using this framework, we have crafted a diagnostic benchmark consisting of 374 original images and 1,122 high-quality question-answer (QA) pairs. The benchmark covers two aspects of conflict and three question types, providing a thorough assessment tool. We apply this benchmark to assess the conflict-resolution capabilities of nine representative MLLMs from various model families. Our results indicate an evident over-reliance on parametric knowledge for approximately 20% of all queries, especially among Yes-No and action-related problems. Based on these findings, we evaluate the effectiveness of existing approaches to mitigating the conflicts and compare them to our "Focus-on-Vision" prompting strategy. Despite some improvement, the vision-knowledge conflict remains unresolved and can be further scaled through our data construction framework. Our proposed framework, benchmark, and analysis contribute to the understanding and mitigation of vision-knowledge conflicts in MLLMs.