Challenges & Opportunities with LLM-Assisted Visualization Retargeting

TL;DR

This paper evaluates how Large Language Models can assist in automatically retargeting visualizations to new datasets, analyzing their performance, limitations, and design considerations for future systems.

Contribution

It compares two LLM-based approaches for visualization retargeting and provides insights into their effectiveness and failure modes.

Findings

Both approaches struggle with untransformed new data.

Structural guidance improves code generation accuracy.

Identifies key challenges and design recommendations for LLM-assisted retargeting.

Abstract

Despite the ubiquity of visualization examples published on the web, retargeting existing custom chart implementations to new datasets remains difficult, time-intensive, and tedious. The adaptation process assumes author familiarity with both the implementation of the example as well as how the new dataset might need to be transformed to fit into the example code. With recent advances in Large Language Models (LLMs), automatic adaptation of code can be achieved from high-level user prompts, reducing the barrier for visualization retargeting. To better understand how LLMs can assist retargeting and its potential limitations, we characterize and evaluate the performance of LLM assistance across multiple datasets and charts of varying complexity, categorizing failures according to type and severity. In our evaluation, we compare two approaches: (1) directly instructing the LLM model to fully generate and adapt code by treating code as text inputs and (2) a more constrained program synthesis pipeline where the LLM guides the code construction process by providing structural information (e.g., visual encodings) based on properties of the example code and data. We find that both approaches struggle when new data has not been appropriately transformed, and discuss important design recommendations for future retargeting systems.