Chart-to-Text: A Large-Scale Benchmark for Chart Summarization
Shankar Kantharaj, Rixie Tiffany Ko Leong, Xiang Lin, Ahmed Masry,, Megh Thakkar, Enamul Hoque, Shafiq Joty

TL;DR
This paper introduces Chart-to-text, a large-scale benchmark with datasets and models for generating natural language summaries from various types of charts, highlighting current challenges and future directions.
Contribution
It provides a comprehensive benchmark with datasets and baseline models for chart summarization, addressing both data table and image-based chart understanding.
Findings
Models produce fluent summaries with reasonable BLEU scores.
Hallucinations and factual errors are common in generated summaries.
Difficulty in explaining complex patterns remains a challenge.
Abstract
Charts are commonly used for exploring data and communicating insights. Generating natural language summaries from charts can be very helpful for people in inferring key insights that would otherwise require a lot of cognitive and perceptual efforts. We present Chart-to-text, a large-scale benchmark with two datasets and a total of 44,096 charts covering a wide range of topics and chart types. We explain the dataset construction process and analyze the datasets. We also introduce a number of state-of-the-art neural models as baselines that utilize image captioning and data-to-text generation techniques to tackle two problem variations: one assumes the underlying data table of the chart is available while the other needs to extract data from chart images. Our analysis with automatic and human evaluation shows that while our best models usually generate fluent summaries and yield…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Multimodal Machine Learning Applications
