G-PCGRL: Procedural Graph Data Generation via Reinforcement Learning

TL;DR

G-PCGRL is a reinforcement learning-based method for controllably and efficiently generating graph data structures, such as game economies and skill trees, to assist game designers.

Contribution

It extends the PCGRL framework to handle graph data generation as a Markov decision process with new representations.

Findings

Capable of quick and reliable graph content generation.

Models are controllable in node type and quantity.

Outperforms random search and evolutionary algorithms.

Abstract

Graph data structures offer a versatile and powerful means to model relationships and interconnections in various domains, promising substantial advantages in data representation, analysis, and visualization. In games, graph-based data structures are omnipresent and represent, for example, game economies, skill trees or complex, branching quest lines. With this paper, we propose G-PCGRL, a novel and controllable method for the procedural generation of graph data using reinforcement learning. Therefore, we frame this problem as manipulating a graph's adjacency matrix to fulfill a given set of constraints. Our method adapts and extends the Procedural Content Generation via Reinforcement Learning (PCGRL) framework and introduces new representations to frame the problem of graph data generation as a Markov decision process. We compare the performance of our method with the original PCGRL, the run time with a random search and evolutionary algorithm, and evaluate G-PCGRL on two graph data domains in games: game economies and skill trees. The results show that our method is capable of generating graph-based content quickly and reliably to support and inspire designers in the game creation process. In addition, trained models are controllable in terms of the type and number of nodes to be generated.