Beyond Description: A Multimodal Agent Framework for Insightful Chart Summarization
Yuhang Bai, Yujuan Ding, Shanru Lin, Wenqi Fan

TL;DR
This paper introduces a novel multi-agent framework that leverages multimodal large language models to generate insightful, deep summaries of charts, surpassing traditional low-level description methods.
Contribution
The paper presents Chart Insight Agent Flow, a new multi-agent approach for chart summarization, and introduces ChartSummInsights, a benchmark dataset with expert-annotated insightful summaries.
Findings
Significant improvement in summarization quality with deep insights.
Effective utilization of MLLMs for complex data interpretation.
Introduction of a new dataset for benchmarking chart summarization.
Abstract
Chart summarization is crucial for enhancing data accessibility and the efficient consumption of information. However, existing methods, including those with Multimodal Large Language Models (MLLMs), primarily focus on low-level data descriptions and often fail to capture the deeper insights which are the fundamental purpose of data visualization. To address this challenge, we propose Chart Insight Agent Flow, a plan-and-execute multi-agent framework effectively leveraging the perceptual and reasoning capabilities of MLLMs to uncover profound insights directly from chart images. Furthermore, to overcome the lack of suitable benchmarks, we introduce ChartSummInsights, a new dataset featuring a diverse collection of real-world charts paired with high-quality, insightful summaries authored by human data analysis experts. Experimental results demonstrate that our method significantly…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHandwritten Text Recognition Techniques · Multimodal Machine Learning Applications · Data Visualization and Analytics
