Supporting Serendipity: Opportunities and Challenges for Human-AI Collaboration in Qualitative Analysis

TL;DR

This paper explores how AI can assist qualitative researchers in analyzing large data sets while respecting their need for control, serendipity, and handling ambiguity, highlighting opportunities and challenges in human-AI collaboration.

Contribution

It provides an in-depth study of qualitative researchers' practices and identifies design directions for AI tools that support human agency and serendipity.

Findings

Researchers value full control over qualitative analysis processes.

AI can support but should not override human decision-making.

Designing AI tools requires balancing assistance with preserving researcher agency.

Abstract

Qualitative inductive methods are widely used in CSCW and HCI research for their ability to generatively discover deep and contextualized insights, but these inherently manual and human-resource-intensive processes are often infeasible for analyzing large corpora. Researchers have been increasingly interested in ways to apply qualitative methods to "big" data problems, hoping to achieve more generalizable results from larger amounts of data while preserving the depth and richness of qualitative methods. In this paper, we describe a study of qualitative researchers' work practices and their challenges, with an eye towards whether this is an appropriate domain for human-AI collaboration and what successful collaborations might entail. Our findings characterize participants' diverse methodological practices and nuanced collaboration dynamics, and identify areas where they might benefit from AI-based tools. While participants highlight the messiness and uncertainty of qualitative inductive analysis, they still want full agency over the process and believe that AI should not interfere. Our study provides a deep investigation of task delegability in human-AI collaboration in the context of qualitative analysis, and offers directions for the design of AI assistance that honor serendipity, human agency, and ambiguity.