Configurable Agent With Reward As Input: A Play-Style Continuum Generation

TL;DR

This paper introduces CARI, a flexible reinforcement learning agent capable of simulating a wide range of play-styles in complex video games using a single training process, improving game testing and balancing.

Contribution

The paper presents CARI, a novel RL agent that models diverse play-styles through reward input, outperforming traditional reward shaping methods with a unified training loop.

Findings

CARI can generate a continuum of play-styles.

CARI outperforms reward shaping baseline in archetype generation.

Single training loop effectively models multiple behaviors.

Abstract

Modern video games are becoming richer and more complex in terms of game mechanics. This complexity allows for the emergence of a wide variety of ways to play the game across the players. From the point of view of the game designer, this means that one needs to anticipate a lot of different ways the game could be played. Machine Learning (ML) could help address this issue. More precisely, Reinforcement Learning is a promising answer to the need of automating video game testing. In this paper we present a video game environment which lets us define multiple play-styles. We then introduce CARI: a Configurable Agent with Reward as Input. An agent able to simulate a wide continuum range of play-styles. It is not constrained to extreme archetypal behaviors like current methods using reward shaping. In addition it achieves this through a single training loop, instead of the usual one loop per play-style. We compare this novel training approach with the more classic reward shaping approach and conclude that CARI can also outperform the baseline on archetypes generation. This novel agent could be used to investigate behaviors and balancing during the production of a video game with a realistic amount of training time.