VProChart: Answering Chart Question through Visual Perception Alignment Agent and Programmatic Solution Reasoning
Muye Huang, Lingling Zhang, Lai Han, Wenjun Wu, Xinyu Zhang, Jun Liu

TL;DR
VProChart is a novel framework that improves chart question answering by aligning visual perception with programmatic reasoning, enabling better interpretation and reasoning over complex chart data.
Contribution
The paper introduces VProChart, combining a visual perception alignment agent with programmatic reasoning to enhance chart understanding and question answering performance.
Findings
Outperforms existing CQA methods on ChartQA and PlotQA datasets
Effectively models chart elements based on human visual perception principles
Enables precise numerical and logical reasoning in chart interpretation
Abstract
Charts are widely used for data visualization across various fields, including education, research, and business. Chart Question Answering (CQA) is an emerging task focused on the automatic interpretation and reasoning of data presented in charts. However, chart images are inherently difficult to interpret, and chart-related questions often involve complex logical and numerical reasoning, which hinders the performance of existing models. This paper introduces VProChart, a novel framework designed to address these challenges in CQA by integrating a lightweight Visual Perception Alignment Agent (VPAgent) and a Programmatic Solution Reasoning approach. VPAgent aligns and models chart elements based on principles of human visual perception, enhancing the understanding of chart context. The Programmatic Solution Reasoning approach leverages large language models (LLMs) to transform natural…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsMultimodal Machine Learning Applications · Intelligent Tutoring Systems and Adaptive Learning · Topic Modeling
