The Truth, the Whole Truth, and Nothing but the Truth: Automatic Visualization Evaluation from Reconstruction Quality

TL;DR

This paper introduces an automated, scalable metric for evaluating visualization quality by measuring how accurately the original data can be reconstructed from the visualization, reducing reliance on costly human assessments.

Contribution

It presents a novel reconstruction-based evaluation method that uses original data as implicit ground truth to assess visualization quality automatically.

Findings

Provides a scalable alternative to human evaluation

Achieves reliable assessment of visualization quality

Reduces evaluation costs and time

Abstract

Recent advances in AI enable the automatic generation of visualizations directly from textual prompts using agentic workflows. However, visualizations produced via one-shot generative methods often suffer from insufficient quality, typically requiring a human in the loop to refine the outputs. Human evaluation, though effective, is costly and impractical at scale. To alleviate this problem, we propose an automated metric that evaluates visualization quality without relying on extensive human-labeled datasets. Instead, our approach uses the original underlying data as implicit ground truth. Specifically, we introduce a method that measures visualization quality by assessing the reconstruction accuracy of the original data from the visualization itself. This reconstruction-based metric provides an autonomous and scalable proxy for thorough human evaluation, facilitating more efficient and reliable AI-driven visualization workflows.