Unveiling the Tapestry of Consistency in Large Vision-Language Models

TL;DR

This paper introduces ConBench, a benchmark for evaluating consistency in large vision-language models across different solution spaces, revealing key insights and proposing diagnostic refinement to improve model reliability.

Contribution

It presents the first multi-modal benchmark ConBench for analyzing LVLM consistency and uncovers relationships between solution space size, accuracy, and model bias, proposing refinement methods.

Findings

Larger solution spaces decrease accuracy in the discriminate realm.

Discriminative question accuracy correlates with caption consistency.

Closed-source models show higher bias in consistency.

Abstract

Large vision-language models (LVLMs) have recently achieved rapid progress, exhibiting great perception and reasoning abilities concerning visual information. However, when faced with prompts in different sizes of solution spaces, LVLMs fail to always give consistent answers regarding the same knowledge point. This inconsistency of answers between different solution spaces is prevalent in LVLMs and erodes trust. To this end, we provide a multi-modal benchmark ConBench, to intuitively analyze how LVLMs perform when the solution space of a prompt revolves around a knowledge point. Based on the ConBench tool, we are the first to reveal the tapestry and get the following findings: (1) In the discriminate realm, the larger the solution space of the prompt, the lower the accuracy of the answers. (2) Establish the relationship between the discriminative and generative realms: the accuracy of the discriminative question type exhibits a strong positive correlation with its Consistency with the caption. (3) Compared to open-source models, closed-source models exhibit a pronounced bias advantage in terms of Consistency. Eventually, we ameliorate the consistency of LVLMs by trigger-based diagnostic refinement, indirectly improving the performance of their caption. We hope this paper will accelerate the research community in better evaluating their models and encourage future advancements in the consistency domain. The project is available at https://github.com/foundation-multimodal-models/ConBench.