AI, Humans, and Data Science: Optimizing Roles Across Workflows and the Workforce

TL;DR

This paper examines how AI can augment data science workflows ethically and effectively, emphasizing human-AI collaboration while warning against over-reliance on automation that may compromise understanding and quality.

Contribution

It introduces a framework for evaluating AI in data science using TBJ principles and discusses the balance between AI assistance and human oversight in research workflows.

Findings

AI can enhance data analysis efficiency and quality.

Over-automation risks reducing understanding and ethical oversight.

Human-machine collaboration is vital for effective data science.

Abstract

AI is transforming research. It is being leveraged to construct surveys, synthesize data, conduct analysis, and write summaries of the results. While the promise is to create efficiencies and increase quality, the reality is not always as clear cut. Leveraging our framework of Truth, Beauty, and Justice (TBJ) which we use to evaluate AI, machine learning and computational models for effective and ethical use (Taber and Timpone 1997; Timpone and Yang 2024), we consider the potential and limitation of analytic, generative, and agentic AI to augment data scientists or take on tasks traditionally done by human analysts and researchers. While AI can be leveraged to assist analysts in their tasks, we raise some warnings about push-button automation. Just as earlier eras of survey analysis created some issues when the increased ease of using statistical software allowed researchers to conduct analyses they did not fully understand, the new AI tools may create similar but larger risks. We emphasize a human-machine collaboration perspective (Daugherty and Wilson 2018) throughout the data science workflow and particularly call out the vital role that data scientists play under VUCA decision areas. We conclude by encouraging the advance of AI tools to complement data scientists but advocate for continued training and understanding of methods to ensure the substantive value of research is fully achieved by applying, interpreting, and acting upon results most effectively and ethically.