Unexplored flaws in multiple-choice VQA evaluations

TL;DR

This paper uncovers previously unexamined biases in prompt formatting that significantly affect the reliability of multiple-choice VQA evaluations across various models and datasets, revealing a need for improved assessment methods.

Contribution

The study identifies three key prompt formatting biases in multiple-choice VQA and demonstrates their impact across seven models and five datasets, highlighting limitations of current bias mitigation strategies.

Findings

Prompt formatting biases significantly influence VQA performance.

Biases persist regardless of answer order or model confidence.

Existing mitigation methods do not address these new biases.

Abstract

Multimodal Large Language Models (MLLMs) demonstrate strong capabilities in handling image-text inputs. A common way to assess this ability is through multiple-choice Visual Question Answering (VQA). Earlier works have already revealed that these benchmarks are sensitive to answer choice order, a limitation that can be mitigated through careful design. Yet, we highlight additional, unexplored biases in prompt formatting that question the reliability of current MLLM evaluations. Specifically, we identify three key variation factors in prompt formatting and analyze their impact through a large-scale study involving $\mathbf{\text{seven}}$ MLLMs and $\mathbf{\text{five}}$ VQA datasets, spanning $\mathbf{48}$ distinct $\mathbf{\text{prompt format variations}}$. Our findings reveal that multiple-choice VQA is highly sensitive to minor prompt format changes, even when these changes are semantically neutral. We further demonstrate that these biases persist independently of known order biases or the MLLM's confidence in the correct answer. Finally, we demonstrate that existing bias mitigation strategies fail to address these newly identified biases.