PlotCraft: Pushing the Limits of LLMs for Complex and Interactive Data Visualization

TL;DR

PlotCraft introduces a comprehensive benchmark and dataset for evaluating and improving large language models' ability to generate complex, interactive data visualizations across diverse domains, addressing current performance gaps.

Contribution

The paper presents PlotCraft, a new benchmark and dataset for complex visualization tasks, and introduces PlotCraftor, a small yet effective code generation model that significantly improves visualization capabilities.

Findings

Existing LLMs perform poorly on complex visualization tasks.

PlotCraftor achieves over 50% performance improvement on hard tasks.

The benchmark and dataset enable systematic evaluation and development of visualization models.

Abstract

Recent Large Language Models (LLMs) have demonstrated remarkable proficiency in code generation. However, their ability to create complex visualizations for scaled and structured data remains largely unevaluated and underdeveloped. To address this gap, we introduce PlotCraft, a new benchmark featuring 1k challenging visualization tasks that cover a wide range of topics, such as finance, scientific research, and sociology. The benchmark is structured around seven high-level visualization tasks and encompasses 48 distinct chart types. Crucially, it is the first to systematically evaluate both single-turn generation and multi-turn refinement across a diverse spectrum of task complexities. Our comprehensive evaluation of 23 leading LLMs on PlotCraft reveals obvious performance deficiencies in handling sophisticated visualization tasks. To bridge this performance gap, we develope SynthVis-30K, a large-scale, high-quality dataset of complex visualization code synthesized via a collaborative agent framework. Building upon this dataset, we develope PlotCraftor, a novel code generation model that achieves strong capabilities in complex data visualization with a remarkably small size. Across VisEval, PandasPlotBench, and our proposed PlotCraft, PlotCraftor shows performance comparable to that of leading proprietary approaches. Especially, on hard task, Our model achieves over 50% performance improvement. We will release the benchmark, dataset, and code at https://github.com/Speakn0w/PlotCraft-Benchmark.