ChartDesign: Towards LLM Designer of Data Visualization

TL;DR

This paper introduces ChartDesign, a system that uses fine-tuned large language models to automate data visualization design, achieving high accuracy and human-preferred results across diverse domains.

Contribution

It presents a novel approach to automate chart design by post-training LLMs on a curated dataset, improving accuracy and generalization over existing methods.

Findings

Achieves up to 84% accuracy in chart design attribute prediction.

Outperforms strong baselines with significant margin.

Generated charts are visually appealing and preferred by humans.

Abstract

Charts are the dominant medium for visualizing data, discovering patterns and trends, and communicating data driven insights, yet designing them still requires expensive human effort and expertise, such as selecting appropriate chart types, axis orientations, font sizes, and layouts. Most automatic visualization systems rely on handcrafted heuristics or simple rule matching and therefore struggle to generalize across domains. This work explores the potential of large language models (LLMs) as chart designers. We propose ChartDesign, which post-trains LLMs to imitate human experts and generate chart design attributes given tabular data. To this end, we curate a diverse training corpus of data design pairs from charts in public surveys (PewResearch) and academic repositories (CharXiV). Vision language models are used to extract data and design attributes from these charts, including chart type, sub type, alignment, titles, axis labels, and bar spacing, formatted as JSON. We then fine tune LoRA adapters on Phi3, Qwen3, and InternVL2.5 to learn a mapping from data to design specifications. ChartDesign significantly improves chart design performance over strong baselines, achieving up to 84% accuracy on a held-out test set (vs. 53% for the best baseline) and generalizing to unseen domains. We further show that charts rendered from ChartDesign generated specifications are visually appealing and human preferred, narrowing the human AI gap in data visualization.