More Than "Means to an End": Supporting Reasoning with Transparently Designed AI Data Science Processes

TL;DR

This paper explores how transparent intermediate artifacts in AI data science workflows enhance reasoning, evaluation, and reformulation of problems, especially in high-stakes medical domains, by supporting user understanding and decision-making.

Contribution

It introduces the concept of designing AI workflows with transparent intermediate artifacts to improve reasoning and user engagement in complex data science tasks.

Findings

Intermediate artifacts aid users in reasoning about analytical choices.

Transparent design helps users refine questions and incorporate their knowledge.

Success depends on constructing workflows around readable, interpretable components.

Abstract

Generative artificial intelligence (AI) tools can now help people perform complex data science tasks regardless of their expertise. While these tools have great potential to help more people work with data, their end-to-end approach does not support users in evaluating alternative approaches and reformulating problems, both critical to solving open-ended tasks in high-stakes domains. In this paper, we reflect on two AI data science systems designed for the medical setting and how they function as tools for thought. We find that success in these systems was driven by constructing AI workflows around intentionally-designed intermediate artifacts, such as readable query languages, concept definitions, or input-output examples. Despite opaqueness in other parts of the AI process, these intermediates helped users reason about important analytical choices, refine their initial questions, and contribute their unique knowledge. We invite the HCI community to consider when and how intermediate artifacts should be designed to promote effective data science thinking.