Characterising the Creative Process in Humans and Large Language Models

TL;DR

This paper introduces an automated method to analyze the creative process of humans and large language models using semantic space exploration, revealing differences in search strategies and their relation to creativity scores.

Contribution

It presents a novel automated approach to characterize the creative process in LLMs and humans, contrasting their exploration strategies during creative tasks.

Findings

Humans show both persistent and flexible search pathways to creativity.

LLMs tend to be biased towards either persistent or flexible paths, varying by task.

More flexible LLMs tend to have higher creativity scores.

Abstract

Large language models appear quite creative, often performing on par with the average human on creative tasks. However, research on LLM creativity has focused solely on \textit{products}, with little attention on the creative \textit{process}. Process analyses of human creativity often require hand-coded categories or exploit response times, which do not apply to LLMs. We provide an automated method to characterise how humans and LLMs explore semantic spaces on the Alternate Uses Task, and contrast with behaviour in a Verbal Fluency Task. We use sentence embeddings to identify response categories and compute semantic similarities, which we use to generate jump profiles. Our results corroborate earlier work in humans reporting both persistent (deep search in few semantic spaces) and flexible (broad search across multiple semantic spaces) pathways to creativity, where both pathways lead to similar creativity scores. LLMs were found to be biased towards either persistent or flexible paths, that varied across tasks. Though LLMs as a population match human profiles, their relationship with creativity is different, where the more flexible models score higher on creativity. Our dataset and scripts are available on \href{https://github.com/surabhisnath/Creative_Process}{GitHub}.