Understanding Why ChatGPT Outperforms Humans in Visualization Design Advice

TL;DR

This study analyzes why ChatGPT models outperform humans in visualization advice, highlighting differences in response structure, knowledge, and quality, and discusses implications for AI-enhanced user experiences.

Contribution

It provides a systematic comparison of ChatGPT and human responses in visualization tasks, revealing model strengths and informing future AI-human collaboration.

Findings

ChatGPT-4 shows hybrid characteristics of earlier models and humans.

Models are generally preferred over humans for visualization advice.

Coverage and technical focus contribute to higher response quality.

Abstract

This paper investigates why recent generative AI models outperform humans in data visualization knowledge tasks. Through systematic comparative analysis of responses to visualization questions, we find that differences exist between two ChatGPT models and human outputs over rhetorical structure, knowledge breadth, and perceptual quality. Our findings reveal that ChatGPT-4, as a more advanced model, displays a hybrid of characteristics from both humans and ChatGPT-3.5. The two models were generally favored over human responses, while their strengths in coverage and breadth, and emphasis on technical and task-oriented visualization feedback collectively shaped higher overall quality. Based on our findings, we draw implications for advancing user experiences based on the potential of LLMs and human perception over their capabilities, with relevance to broader applications of AI.