Unsolvable Problem Detection: Robust Understanding Evaluation for Large Multimodal Models

TL;DR

This paper proposes Unsolvable Problem Detection (UPD), a new evaluation task for Large Multimodal Models to assess their true understanding by identifying when they should withhold answers, revealing limitations of current benchmarks.

Contribution

Introduces UPD as a novel task and MM-UPD Bench as a benchmark to evaluate LMMs' ability to recognize unsolvable problems, highlighting gaps in current understanding assessments.

Findings

Most LMMs struggle with UPD tasks, indicating overestimation of their understanding.

Chain-of-thought and self-reflection improve LMM performance on UPD.

Current benchmarks do not adequately measure models' trustworthiness and true understanding.

Abstract

This paper introduces a novel task to evaluate the robust understanding capability of Large Multimodal Models (LMMs), termed $\textbf{Unsolvable Problem Detection (UPD)}$. Multiple-choice question answering (MCQA) is widely used to assess the understanding capability of LMMs, but it does not guarantee that LMMs truly comprehend the answer. UPD assesses the LMM's ability to withhold answers when encountering unsolvable problems of MCQA, verifying whether the model truly understands the answer. UPD encompasses three problems: Absent Answer Detection (AAD), Incompatible Answer Set Detection (IASD), and Incompatible Visual Question Detection (IVQD), covering unsolvable cases like answer-lacking or incompatible choices and image-question mismatches. For the evaluation, we introduce the MM-UPD Bench, a benchmark for assessing performance across various ability dimensions. Our experiments reveal that even most LMMs, which demonstrate adequate performance on existing benchmarks, struggle significantly with MM-UPD, underscoring a novel aspect of trustworthiness that current benchmarks have overlooked. A detailed analysis shows that LMMs have different bottlenecks and chain-of-thought and self-reflection improved performance for LMMs with the bottleneck in their LLM capability. We hope our insights will enhance the broader understanding and development of more reliable LMMs. The code is available at https://github.com/AtsuMiyai/UPD.