Chart-FR1: Visual Focus-Driven Fine-Grained Reasoning on Dense Charts

TL;DR

Chart-FR1 introduces a focus-driven reasoning model that enhances fine-grained perception and adaptive reasoning in dense charts, outperforming existing models on a new challenging benchmark.

Contribution

The paper proposes a novel focus-driven reasoning framework with visual focusing chain-of-thought and reinforcement learning, addressing perception and reasoning challenges in high-density charts.

Findings

Outperforms state-of-the-art models on multiple chart benchmarks.

Effectively compresses redundant visual information for better focusing.

Achieves superior reasoning accuracy on the new HID-Chart benchmark.

Abstract

Multimodal large language models (MLLMs) have shown considerable potential in chart understanding and reasoning tasks. However, they still struggle with high information density (HID) charts characterized by multiple subplots, legends, and dense annotations due to three major challenges: (1) limited fine-grained perception results in the omission of critical visual cues; (2) redundant or noisy visual information undermines the performance of multimodal reasoning; (3) lack of adaptive deep reasoning relative to the amount of visual information. To tackle these challenges, we present a novel focus-driven fine-grained chart reasoning model, Chart-FR1, to improve perception, focusing efficiency, and adaptive deep reasoning on HID charts. Specifically, we propose Focus-CoT, a visual focusing chain-of-thought that enhances fine-grained perception by explicitly linking reasoning steps to key visual cues, such as local image regions and OCR signals. Building on this, we introduce Focus-GRPO, a focus-driven reinforcement learning algorithm with an information-efficiency reward that compresses redundant visual information for efficient focusing, and an adaptive KL penalty mechanism that enables flexible control over reasoning depth as more visual cues are discovered. Furthermore, to fill the gap in benchmarks for HID charts, we build HID-Chart, a challenging benchmark with an information-density metric designed to evaluate fine-grained chart reasoning capabilities. Extensive experiments on multiple chart benchmarks demonstrate that Chart-FR1 outperforms state-of-the-art MLLMs in chart understanding and reasoning. Code is available at https://github.com/phkhub/Chart-FR1.