ChartGalaxy: A Dataset for Infographic Chart Understanding and Generation

TL;DR

ChartGalaxy is a large-scale dataset that enhances the understanding and generation of complex infographic charts by capturing their visual and structural diversity, aiding vision-language models in multimodal reasoning.

Contribution

The paper introduces ChartGalaxy, a million-scale dataset with synthetic infographic charts, enabling improved understanding, benchmarking, and generation of complex visual data.

Findings

Enhanced infographic chart understanding through fine-tuning.

Benchmarking code generation for infographic charts.

Facilitated example-based infographic chart generation.

Abstract

Infographic charts are a powerful medium for communicating abstract data by combining visual elements (e.g., charts, images) with textual information. However, their visual and structural richness poses challenges for large vision-language models (LVLMs), which are typically trained on plain charts. To bridge this gap, we introduce ChartGalaxy, a million-scale dataset designed to advance the understanding and generation of infographic charts. The dataset is constructed through an inductive process that identifies 75 chart types, 440 chart variations, and 68 layout templates from real infographic charts and uses them to create synthetic ones programmatically. We showcase the utility of this dataset through: 1) improving infographic chart understanding via fine-tuning, 2) benchmarking code generation for infographic charts, and 3) enabling example-based infographic chart generation. By capturing the visual and structural complexity of real design, ChartGalaxy provides a useful resource for enhancing multimodal reasoning and generation in LVLMs.