Estimating player completion rate in mobile puzzle games using reinforcement learning

TL;DR

This paper explores using reinforcement learning agents to estimate player completion rates in mobile puzzle games by analyzing the agent's performance and its correlation with human player data.

Contribution

It demonstrates that RL agent performance, particularly the number of moves in top runs, correlates with human completion rates, offering a new method to estimate player difficulty.

Findings

Agent's move count predicts human completion rates.

Behavioral differences between levels are correlated across humans and agents.

Agent performance can be used to model player metrics despite not reaching human-level performance.

Abstract

In this work we investigate whether it is plausible to use the performance of a reinforcement learning (RL) agent to estimate the difficulty measured as the player completion rate of different levels in the mobile puzzle game Lily's Garden.For this purpose we train an RL agent and measure the number of moves required to complete a level. This is then compared to the level completion rate of a large sample of real players.We find that the strongest predictor of player completion rate for a level is the number of moves taken to complete a level of the ~5% best runs of the agent on a given level. A very interesting observation is that, while in absolute terms, the agent is unable to reach human-level performance across all levels, the differences in terms of behaviour between levels are highly correlated to the differences in human behaviour. Thus, despite performing sub-par, it is still possible to use the performance of the agent to estimate, and perhaps further model, player metrics.