A modular framework for automated evaluation of procedural content generation in serious games with deep reinforcement learning agents

TL;DR

This paper introduces a framework for automatically evaluating procedural content generation in serious games using deep reinforcement learning agents, demonstrating its effectiveness through experiments with different NPC generation methods.

Contribution

The study presents a novel automated evaluation framework for PCG in serious games, integrating DRL agents and validating it with diverse NPC generation techniques.

Findings

DRL agents trained on genetic algorithm-based NPCs outperform random NPCs in win rate and training efficiency.

Versions 2 and 3 achieved up to 97% win rate, significantly higher than Version 1.

The framework effectively produces meaningful data for PCG evaluation in serious games.

Abstract

Serious Games (SGs) are nowadays shifting focus to include procedural content generation (PCG) in the development process as a means of offering personalized and enhanced player experience. However, the development of a framework to assess the impact of PCG techniques when integrated into SGs remains particularly challenging. This study proposes a methodology for automated evaluation of PCG integration in SGs, incorporating deep reinforcement learning (DRL) game testing agents. To validate the proposed framework, a previously introduced SG featuring card game mechanics and incorporating three different versions of PCG for nonplayer character (NPC) creation has been deployed. Version 1 features random NPC creation, while versions 2 and 3 utilize a genetic algorithm approach. These versions are used to test the impact of different dynamic SG environments on the proposed framework's agents. The obtained results highlight the superiority of the DRL game testing agents trained on Versions 2 and 3 over those trained on Version 1 in terms of win rate (i.e. number of wins per played games) and training time. More specifically, within the execution of a test emulating regular gameplay, both Versions 2 and 3 peaked at a 97% win rate and achieved statistically significant higher (p=0009) win rates compared to those achieved in Version 1 that peaked at 94%. Overall, results advocate towards the proposed framework's capability to produce meaningful data for the evaluation of procedurally generated content in SGs.