Generative Personas That Behave and Experience Like Humans

TL;DR

This paper introduces a novel approach to creating AI agents that mimic not only human gameplay behavior but also human experience, using reinforcement learning to generate more realistic and diverse player personas in games.

Contribution

It extends procedural personas to include player experience, employing Go-Explore reinforcement learning to produce human-like agents that reflect both behavior and experience.

Findings

Generated agents exhibit distinctive play styles.

Agents replicate human experience responses.

Experience data enhances behavioral exploration.

Abstract

Using artificial intelligence (AI) to automatically test a game remains a critical challenge for the development of richer and more complex game worlds and for the advancement of AI at large. One of the most promising methods for achieving that long-standing goal is the use of generative AI agents, namely procedural personas, that attempt to imitate particular playing behaviors which are represented as rules, rewards, or human demonstrations. All research efforts for building those generative agents, however, have focused solely on playing behavior which is arguably a narrow perspective of what a player actually does in a game. Motivated by this gap in the existing state of the art, in this paper we extend the notion of behavioral procedural personas to cater for player experience, thus examining generative agents that can both behave and experience their game as humans would. For that purpose, we employ the Go-Explore reinforcement learning paradigm for training human-like procedural personas, and we test our method on behavior and experience demonstrations of more than 100 players of a racing game. Our findings suggest that the generated agents exhibit distinctive play styles and experience responses of the human personas they were designed to imitate. Importantly, it also appears that experience, which is tied to playing behavior, can be a highly informative driver for better behavioral exploration.