Improving Playtesting Coverage via Curiosity Driven Reinforcement Learning Agents

TL;DR

This paper presents curiosity-driven reinforcement learning agents that autonomously explore complex 3D game scenarios to improve testing coverage, identify issues, and assist game design decisions, addressing limitations of previous exploration methods.

Contribution

It introduces a curiosity-based reinforcement learning approach for autonomous game testing that effectively explores complex environments and enhances coverage compared to prior techniques.

Findings

Agents learn complex navigation in 3D environments

Curiosity-driven exploration improves coverage of game scenarios

Visualization strategies aid in identifying design issues

Abstract

As modern games continue growing both in size and complexity, it has become more challenging to ensure that all the relevant content is tested and that any potential issue is properly identified and fixed. Attempting to maximize testing coverage using only human participants, however, results in a tedious and hard to orchestrate process which normally slows down the development cycle. Complementing playtesting via autonomous agents has shown great promise accelerating and simplifying this process. This paper addresses the problem of automatically exploring and testing a given scenario using reinforcement learning agents trained to maximize game state coverage. Each of these agents is rewarded based on the novelty of its actions, thus encouraging a curious and exploratory behaviour on a complex 3D scenario where previously proposed exploration techniques perform poorly. The curious agents are able to learn the complex navigation mechanics required to reach the different areas around the map, thus providing the necessary data to identify potential issues. Moreover, the paper also explores different visualization strategies and evaluates how to make better use of the collected data to drive design decisions and to recognize possible problems and oversights.