Designing Human and Generative AI Collaboration

TL;DR

This study investigates how different human-AI collaboration designs impact creative writing, showing AI assistance boosts productivity but collaboration structure affects quality, satisfaction, and diversity.

Contribution

It provides empirical insights into how collaboration design influences creative outcomes and user experience in human-AI creative partnerships.

Findings

AI assistance improves productivity across models

Collaboration design affects output quality and satisfaction

Early human involvement mitigates diversity loss

Abstract

We examined the effectiveness of various human-AI collaboration designs on creative work. Through a human subjects experiment set in the context of creative writing, we found that while AI assistance improved productivity across all models, collaboration design significantly influenced output quality, user satisfaction, and content characteristics. Models incorporating human creative input delivered higher content interestingness and overall quality as well as greater task performer satisfaction compared to conditions where humans were limited to confirming AI's output. Increased AI involvement encouraged creators to explore beyond personal experience but also led to lower aggregate diversity in stories and genres among participants. However, this effect was mitigated through human participation in early creative tasks. These findings underscore the importance of preserving the human creative role to ensure quality, satisfaction, and creative diversity in human-AI collaboration.