The Qualitative Laboratory: Theory Prototyping and Hypothesis Generation with Large Language Models

TL;DR

This paper introduces a novel sociological simulation method using Large Language Models to generate rich, nuanced hypotheses about social group interpretations, surpassing traditional qualitative and rule-based modeling approaches.

Contribution

It presents a new 'qualitative laboratory' approach leveraging LLMs for sociological hypothesis generation, addressing limitations of existing methods.

Findings

Generated nuanced hypotheses about social group reactions to policies.

Demonstrated counter-intuitive insights challenging existing theories.

Proposed a workflow integrating simulation with empirical validation.

Abstract

A central challenge in social science is to generate rich qualitative hypotheses about how diverse social groups might interpret new information. This article introduces and illustrates a novel methodological approach for this purpose: sociological persona simulation using Large Language Models (LLMs), which we frame as a "qualitative laboratory". We argue that for this specific task, persona simulation offers a distinct advantage over established methods. By generating naturalistic discourse, it overcomes the lack of discursive depth common in vignette surveys, and by operationalizing complex worldviews through natural language, it bypasses the formalization bottleneck of rule-based agent-based models (ABMs). To demonstrate this potential, we present a protocol where personas derived from a sociological theory of climate reception react to policy messages. The simulation produced nuanced and counter-intuitive hypotheses - such as a conservative persona's rejection of a national security frame - that challenge theoretical assumptions. We conclude that this method, used as part of a "simulation then validation" workflow, represents a superior tool for generating deeply textured hypotheses for subsequent empirical testing.