How Do Analysts Understand and Verify AI-Assisted Data Analyses?

TL;DR

This study investigates how data analysts understand and verify AI-generated analyses, revealing common workflows and offering design recommendations to improve AI-assistant tools for data verification.

Contribution

The paper presents a qualitative study of analyst verification behaviors using a novel design probe, providing insights and recommendations for enhancing AI-assisted data analysis tools.

Findings

Analysts employ diverse verification strategies.

Background influences verification approaches.

Design insights for improving AI-assistant tools.

Abstract

Data analysis is challenging as it requires synthesizing domain knowledge, statistical expertise, and programming skills. Assistants powered by large language models (LLMs), such as ChatGPT, can assist analysts by translating natural language instructions into code. However, AI-assistant responses and analysis code can be misaligned with the analyst's intent or be seemingly correct but lead to incorrect conclusions. Therefore, validating AI assistance is crucial and challenging. Here, we explore how analysts understand and verify the correctness of AI-generated analyses. To observe analysts in diverse verification approaches, we develop a design probe equipped with natural language explanations, code, visualizations, and interactive data tables with common data operations. Through a qualitative user study (n=22) using this probe, we uncover common behaviors within verification workflows and how analysts' programming, analysis, and tool backgrounds reflect these behaviors. Additionally, we provide recommendations for analysts and highlight opportunities for designers to improve future AI-assistant experiences.