InfoChartQA: A Benchmark for Multimodal Question Answering on Infographic Charts
Tianchi Xie, Minzhi Lin, Mengchen Liu, Yilin Ye, Changjian Chen, Shixia Liu

TL;DR
This paper introduces InfoChartQA, a comprehensive benchmark for evaluating multimodal large language models on infographic chart understanding, emphasizing visual recognition and reasoning with paired plain and infographic charts.
Contribution
It provides a new dataset with paired charts and visual-element questions, enabling detailed evaluation and analysis of MLLMs' capabilities in infographic comprehension.
Findings
20 MLLMs show significant performance drops on infographic charts.
Performance is especially poor on visual-element questions involving metaphors.
Paired charts facilitate detailed error analysis and model improvement insights.
Abstract
Understanding infographic charts with design-driven visual elements (e.g., pictograms, icons) requires both visual recognition and reasoning, posing challenges for multimodal large language models (MLLMs). However, existing visual-question answering benchmarks fall short in evaluating these capabilities of MLLMs due to the lack of paired plain charts and visual-element-based questions. To bridge this gap, we introduce InfoChartQA, a benchmark for evaluating MLLMs on infographic chart understanding. It includes 5,642 pairs of infographic and plain charts, each sharing the same underlying data but differing in visual presentations. We further design visual-element-based questions to capture their unique visual designs and communicative intent. Evaluation of 20 MLLMs reveals a substantial performance decline on infographic charts, particularly for visual-element-based questions related to…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsSpeech and dialogue systems · Topic Modeling · Natural Language Processing Techniques
