$C^2$: Scalable Auto-Feedback for LLM-based Chart Generation

TL;DR

This paper introduces C$^2$, a scalable, reference-free framework for improving LLM-based chart generation through automatic feedback and a diverse dataset, significantly enhancing data quality and user preference.

Contribution

The paper presents a novel automatic feedback generator and a large, diverse dataset for LLM chart generation, reducing reliance on costly human curation and improving output quality.

Findings

74% of respondents preferred post-feedback results

ChartAF outperforms nine baselines in experiments

Dataset diversity increased by up to 5982%

Abstract

Generating high-quality charts with Large Language Models (LLMs) presents significant challenges due to limited data and the high cost of scaling through human curation. $\langle \text{instruction}, \text{data}, \text{code} \rangle$ triplets are scarce and expensive to manually curate as their creation demands technical expertise. To address this scalability challenge, we introduce a reference-free automatic feedback generator, which eliminates the need for costly human intervention. Our novel framework, C$^2$, consists of (1) an automatic feedback provider (ChartAF) and (2) a diverse, reference-free dataset (ChartUIE-8K). The results are compelling: in our first experiment, 74% of respondents strongly preferred, and 10% preferred, the results after feedback. The second post-feedback experiment demonstrates that ChartAF outperform nine baselines. Moreover, ChartUIE-8K significantly improves data diversity by increasing queries, datasets, and chart types by 5982%, 1936%, and 91%, respectively, over benchmarks. Finally, a study of LLM users revealed that 94% of participants preferred ChartUIE-8K's queries, with 93% deeming them aligned with real-world use cases. Core contributions are available as open-source at chartsquared.github.io, with ample qualitative examples.