De-rendering, Reasoning, and Repairing Charts with Vision-Language Models

TL;DR

This paper presents a framework that reconstructs charts from images, analyzes their design flaws using vision-language reasoning, and provides actionable recommendations to improve visualization quality, thereby enhancing visualization literacy and tool accessibility.

Contribution

It introduces a novel system combining chart de-rendering, automated flaw detection, and iterative improvement with visualization principles, which was not previously available.

Findings

Generated 10,452 design recommendations on 1,000 charts

Recommendations clustered into 10 categories like axis formatting and color accessibility

Demonstrated the system's potential for principle-based, structured feedback

Abstract

Data visualizations are central to scientific communication, journalism, and everyday decision-making, yet they are frequently prone to errors that can distort interpretation or mislead audiences. Rule-based visualization linters can flag violations, but they miss context and do not suggest meaningful design changes. Directly querying general-purpose LLMs about visualization quality is unreliable: lacking training to follow visualization design principles, they often produce inconsistent or incorrect feedback. In this work, we introduce a framework that combines chart de-rendering, automated analysis, and iterative improvement to deliver actionable, interpretable feedback on visualization design. Our system reconstructs the structure of a chart from an image, identifies design flaws using vision-language reasoning, and proposes concrete modifications supported by established principles in visualization research. Users can selectively apply these improvements and re-render updated figures, creating a feedback loop that promotes both higher-quality visualizations and the development of visualization literacy. In our evaluation on 1,000 charts from the Chart2Code benchmark, the system generated 10,452 design recommendations, which clustered into 10 coherent categories (e.g., axis formatting, color accessibility, legend consistency). These results highlight the promise of LLM-driven recommendation systems for delivering structured, principle-based feedback on visualization design, opening the door to more intelligent and accessible authoring tools.