CCPT: Automatic Gameplay Testing and Validation with Curiosity-Conditioned Proximal Trajectories

TL;DR

This paper introduces CCPT, a deep reinforcement learning method that combines curiosity and imitation learning to automatically explore complex 3D environments, detect gameplay issues, and assist game designers in identifying design oversights.

Contribution

The paper presents a novel curiosity-conditioned RL algorithm that effectively explores complex 3D environments and detects gameplay issues, outperforming baseline methods.

Findings

Higher coverage and bug discovery than baseline methods

Effective in complex 3D navigation environments

Assists game designers in automatic issue detection

Abstract

This paper proposes a novel deep reinforcement learning algorithm to perform automatic analysis and detection of gameplay issues in complex 3D navigation environments. The Curiosity-Conditioned Proximal Trajectories (CCPT) method combines curiosity and imitation learning to train agents to methodically explore in the proximity of known trajectories derived from expert demonstrations. We show how CCPT can explore complex environments, discover gameplay issues and design oversights in the process, and recognize and highlight them directly to game designers. We further demonstrate the effectiveness of the algorithm in a novel 3D navigation environment which reflects the complexity of modern AAA video games. Our results show a higher level of coverage and bug discovery than baselines methods, and it hence can provide a valuable tool for game designers to identify issues in game design automatically.