Exploring the Human-LLM Synergy in Advancing Theory-driven Qualitative Analysis

TL;DR

This paper introduces CHALET, a novel human-LLM collaborative approach that enhances theory-driven qualitative analysis by enabling iterative coding, disagreement analysis, and insight generation, demonstrated through mental health stigma conversations.

Contribution

The paper presents CHALET, a new method for human-LLM collaboration in qualitative analysis, facilitating iterative coding and insight discovery beyond initial theoretical frameworks.

Findings

Human-LLM collaboration uncovers latent stigma themes.

CHALET improves qualitative analysis efficiency.

Insights extend understanding of mental health stigma.

Abstract

Qualitative coding is a demanding yet crucial research method in the field of Human-Computer Interaction (HCI). While recent studies have shown the capability of large language models (LLMs) to perform qualitative coding within theoretical frameworks, their potential for collaborative human-LLM discovery and generation of new insights beyond initial theory remains underexplored. To bridge this gap, we proposed CHALET, a novel approach that harnesses the power of human-LLM partnership to advance theory-driven qualitative analysis by facilitating iterative coding, disagreement analysis, and conceptualization of qualitative data. We demonstrated CHALET's utility by applying it to the qualitative analysis of conversations related to mental-illness stigma, using the attribution model as the theoretical framework. Results highlighted the unique contribution of human-LLM collaboration in uncovering latent themes of stigma across the cognitive, emotional, and behavioral dimensions. We discuss the methodological implications of the human-LLM collaborative approach to theory-based qualitative analysis for the HCI community and beyond.