ChartCheck: Explainable Fact-Checking over Real-World Chart Images
Mubashara Akhtar, Nikesh Subedi, Vivek Gupta, Sahar Tahmasebi, Oana, Cocarascu, Elena Simperl

TL;DR
ChartCheck introduces a large dataset for explainable fact-checking of real-world charts, addressing a gap in misinformation detection in data visualizations, and evaluates vision-language models on this task.
Contribution
The paper presents ChartCheck, a new dataset for fact-checking charts, and provides baseline models and analysis of challenges in chart reasoning.
Findings
Vision-language models struggle with certain chart reasoning types.
The dataset enables systematic evaluation of fact-checking on visual data.
Challenges identified in visual attributes affecting model performance.
Abstract
Whilst fact verification has attracted substantial interest in the natural language processing community, verifying misinforming statements against data visualizations such as charts has so far been overlooked. Charts are commonly used in the real-world to summarize and communicate key information, but they can also be easily misused to spread misinformation and promote certain agendas. In this paper, we introduce ChartCheck, a novel, large-scale dataset for explainable fact-checking against real-world charts, consisting of 1.7k charts and 10.5k human-written claims and explanations. We systematically evaluate ChartCheck using vision-language and chart-to-table models, and propose a baseline to the community. Finally, we study chart reasoning types and visual attributes that pose a challenge to these models
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAnomaly Detection Techniques and Applications · Topic Modeling · Explainable Artificial Intelligence (XAI)
