BigCharts-R1: Enhanced Chart Reasoning with Visual Reinforcement Finetuning

TL;DR

This paper introduces BigCharts-R1, a state-of-the-art chart reasoning model trained on a new diverse dataset and enhanced with reinforcement learning, significantly improving understanding and answering accuracy across various chart types.

Contribution

The paper presents BigCharts, a novel dataset creation pipeline with real-world charts, and a training framework combining supervised fine-tuning with reinforcement learning for improved chart comprehension.

Findings

Outperforms existing models on multiple chart QA benchmarks.

Achieves higher robustness and generalization across diverse chart styles.

Demonstrates effectiveness of reinforcement learning with novel reward signals.

Abstract

Charts are essential to data analysis, transforming raw data into clear visual representations that support human decision-making. Although current vision-language models (VLMs) have made significant progress, they continue to struggle with chart comprehension due to training on datasets that lack diversity and real-world authenticity, or on automatically extracted underlying data tables of charts, which can contain numerous estimation errors. Furthermore, existing models only rely on supervised fine-tuning using these low-quality datasets, severely limiting their effectiveness. To address these issues, we first propose BigCharts, a dataset creation pipeline that generates visually diverse chart images by conditioning the rendering process on real-world charts sourced from multiple online platforms. Unlike purely synthetic datasets, BigCharts incorporates real-world data, ensuring authenticity and visual diversity, while still retaining accurate underlying data due to our proposed replotting process. Additionally, we introduce a comprehensive training framework that integrates supervised fine-tuning with Group Relative Policy Optimization (GRPO)-based reinforcement learning. By introducing novel reward signals specifically designed for chart reasoning, our approach enhances model robustness and generalization across diverse chart styles and domains, resulting in a state-of-the-art chart reasoning model, BigCharts-R1. Extensive experiments demonstrate that our models surpass existing methods on multiple chart question-answering benchmarks compared to even larger open-source and closed-source models.