Playing the Imitation Game: How Perceived Generated Content Shapes Player Experience

TL;DR

This study explores how players' perceptions of whether game levels are human-made or AI-generated influence their experience, revealing biases that affect enjoyment and challenge levels.

Contribution

It demonstrates that players' beliefs about content origin significantly impact their gameplay experience, highlighting perception biases in AI-generated game content.

Findings

Players cannot reliably identify content creators.

Perceived human-made levels are rated more fun and aesthetic.

Perceived AI-generated levels are seen as more frustrating and challenging.

Abstract

With the fast progress of generative AI in recent years, more games are integrating generated content, raising questions regarding how players perceive and respond to this content. To investigate, we ran a mixed-method survey on the games Super Mario Bros. and Sokoban, comparing procedurally generated levels and levels designed by humans to explore how perceptions of the creator relate to players' overall experience of gameplay. Players could not reliably identify the level's creator, yet their experiences were strongly linked to their beliefs about that creator rather than the actual truth. Levels believed to be human-made were rated as more fun and aesthetically pleasing. In contrast, those believed to be AI-generated were rated as more frustrating and challenging. This negative bias appeared spontaneously without knowing the levels' creator and often was based on unreliable cues of "human-likeness." Our results underscore the importance of understanding perception biases when integrating generative systems into games.