ChartAnchor: Chart Grounding with Structural-Semantic Fidelity

TL;DR

ChartAnchor introduces a comprehensive benchmark for evaluating multimodal models' ability to understand and generate structured chart information, addressing limitations of previous narrow and incomplete assessments.

Contribution

The paper presents ChartAnchor, a large-scale, diverse benchmark with novel tasks and a multi-level evaluation framework for holistic chart grounding assessment.

Findings

MLLMs show limitations in numerical precision and code synthesis.

The benchmark reveals gaps in models' structural and content understanding.

Extensive experiments highlight the need for improved structured reasoning.

Abstract

Recent advances in multimodal large language models (MLLMs) highlight the need for benchmarks that rigorously evaluate structured chart comprehension. Chart grounding refers to the bidirectional alignment between a chart's visual appearance and its structured semantics. This task requires models to produce a symbolic specification that faithfully captures the chart's visual and structural intent, while also recovering the underlying tabular data with precise values and relationships. Chart grounding directly reflects a model's capabilities in numerical reasoning, multimodal alignment, and structural reconstruction, and has several important real-world applications. Existing benchmarks, constrained by narrow chart diversity, isolated tasks, and incomplete evaluation frameworks, fail to holistically assess grounding. To address this, we propose ChartAnchor, a comprehensive benchmark of 8k+ chart-table-code triples spanning 30 chart types drawn from diverse real-world and augmented sources. ChartAnchor introduces two complementary tasks: chart-to-code generation and controlled chart-to-table reconstruction, enabling cross-validation of visual and numerical fidelity. A multi-level evaluation framework integrates semantic validation, stylistic analysis, and perceptual metrics to assess both structural and content-level correctness. Extensive experiments on MLLMs reveal critical limitations in numerical precision and code synthesis, emphasizing the need for structured reasoning beyond surface-level perception. By unifying symbolic and data-driven grounding, ChartAnchor establishes a rigorous foundation for chart grounding, offering meaningful insights for advancing MLLMs in scientific, financial, and industrial domains.