Application of predictive machine learning in pen & paper RPG game design

TL;DR

This paper explores machine learning techniques to automate the prediction of opponent levels in pen and paper RPGs, aiming to improve efficiency and consistency over manual methods.

Contribution

It introduces a dedicated dataset, evaluates state-of-the-art ordinal regression methods, and develops a human-inspired benchmark model for level prediction in RPG design.

Findings

Machine learning models outperform manual estimation methods.

A new dataset for RPG level prediction was created.

The human-inspired model provides a reliable benchmark.

Abstract

In recent years, the pen and paper RPG market has experienced significant growth. As a result, companies are increasingly exploring the integration of AI technologies to enhance player experience and gain a competitive edge. One of the key challenges faced by publishers is designing new opponents and estimating their challenge level. Currently, there are no automated methods for determining a monster's level; the only approaches used are based on manual testing and expert evaluation. Although these manual methods can provide reasonably accurate estimates, they are time-consuming and resource-intensive. Level prediction can be approached using ordinal regression techniques. This thesis presents an overview and evaluation of state-of-the-art methods for this task. It also details the construction of a dedicated dataset for level estimation. Furthermore, a human-inspired model was developed to serve as a benchmark, allowing comparison between machine learning algorithms and the approach typically employed by pen and paper RPG publishers. In addition, a specialized evaluation procedure, grounded in domain knowledge, was designed to assess model performance and facilitate meaningful comparisons.