Beyond Winning and Losing: Modeling Human Motivations and Behaviors Using Inverse Reinforcement Learning

TL;DR

This paper introduces Multi-Motivation Behavior Modeling (MMBM), a novel inverse reinforcement learning approach that captures complex human motivations in gameplay, revealing diverse player value structures and behaviors in online games.

Contribution

The paper presents MMBM, a new inverse RL method that models multifaceted human motivations without requiring system dynamics, applicable to complex interactive environments like online games.

Findings

Revealed significant differences in value structures among player groups.

Successfully predicted and explained diverse gameplay behaviors.

Assessed impact of game environment redesign on player motivations.

Abstract

In recent years, reinforcement learning (RL) methods have been applied to model gameplay with great success, achieving super-human performance in various environments, such as Atari, Go, and Poker. However, those studies mostly focus on winning the game and have largely ignored the rich and complex human motivations, which are essential for understanding different players' diverse behaviors. In this paper, we present a novel method called Multi-Motivation Behavior Modeling (MMBM) that takes the multifaceted human motivations into consideration and models the underlying value structure of the players using inverse RL. Our approach does not require the access to the dynamic of the system, making it feasible to model complex interactive environments such as massively multiplayer online games. MMBM is tested on the World of Warcraft Avatar History dataset, which recorded over 70,000 users' gameplay spanning three years period. Our model reveals the significant difference of value structures among different player groups. Using the results of motivation modeling, we also predict and explain their diverse gameplay behaviors and provide a quantitative assessment of how the redesign of the game environment impacts players' behaviors.