Continuous Reinforcement Learning-based Dynamic Difficulty Adjustment in a Visual Working Memory Game

TL;DR

This paper introduces a continuous reinforcement learning approach for dynamic difficulty adjustment in a visual working memory game, improving player experience and performance compared to rule-based methods.

Contribution

It presents a novel continuous RL-based DDA method that effectively manages complex difficulty levels in a VWM game, enhancing player engagement and outcomes.

Findings

Players had higher scores and win rates with the RL approach.

The RL method improved game experience in competence, tension, and affect.

Score decline was less in the RL-based difficulty adjustment.

Abstract

Dynamic Difficulty Adjustment (DDA) is a viable approach to enhance a player's experience in video games. Recently, Reinforcement Learning (RL) methods have been employed for DDA in non-competitive games; nevertheless, they rely solely on discrete state-action space with a small search space. In this paper, we propose a continuous RL-based DDA methodology for a visual working memory (VWM) game to handle the complex search space for the difficulty of memorization. The proposed RL-based DDA tailors game difficulty based on the player's score and game difficulty in the last trial. We defined a continuous metric for the difficulty of memorization. Then, we consider the task difficulty and the vector of difficulty-score as the RL's action and state, respectively. We evaluated the proposed method through a within-subject experiment involving 52 subjects. The proposed approach was compared with two rule-based difficulty adjustment methods in terms of player's score and game experience measured by a questionnaire. The proposed RL-based approach resulted in a significantly better game experience in terms of competence, tension, and negative and positive affect. Players also achieved higher scores and win rates. Furthermore, the proposed RL-based DDA led to a significantly less decline in the score in a 20-trial session.