Comparing Reinforcement Learning and Human Learning using the Game of Hidden Rules

TL;DR

This paper introduces a specialized environment to compare human and reinforcement learning performance across different task structures, highlighting differences and aiding in understanding their respective strengths and weaknesses.

Contribution

It presents a new environment designed specifically for studying how task structure influences human and RL learning, enabling more rigorous comparisons.

Findings

Humans outperform RL in certain structured tasks.

Task structure significantly affects learning performance.

The environment facilitates detailed analysis of human vs. RL learning.

Abstract

Reliable real-world deployment of reinforcement learning (RL) methods requires a nuanced understanding of their strengths and weaknesses and how they compare to those of humans. Human-machine systems are becoming more prevalent and the design of these systems relies on a task-oriented understanding of both human learning (HL) and RL. Thus, an important line of research is characterizing how the structure of a learning task affects learning performance. While increasingly complex benchmark environments have led to improved RL capabilities, such environments are difficult to use for the dedicated study of task structure. To address this challenge we present a learning environment built to support rigorous study of the impact of task structure on HL and RL. We demonstrate the environment's utility for such study through example experiments in task structure that show performance differences between humans and RL algorithms.