Toybox: A Suite of Environments for Experimental Evaluation of Deep Reinforcement Learning

TL;DR

Toybox is a high-performance, open-source suite of Atari environments designed specifically for the experimental evaluation of deep reinforcement learning, enabling new analyses and experiments.

Contribution

The paper introduces Toybox, a new environment suite tailored for deep RL evaluation, addressing the lack of high-quality, testable environments for agent behavior analysis.

Findings

Toybox allows experiments impossible in other environments.

It is high-performance and open-source.

Enables detailed analysis of deep RL agents.

Abstract

Evaluation of deep reinforcement learning (RL) is inherently challenging. In particular, learned policies are largely opaque, and hypotheses about the behavior of deep RL agents are difficult to test in black-box environments. Considerable effort has gone into addressing opacity, but almost no effort has been devoted to producing high quality environments for experimental evaluation of agent behavior. We present TOYBOX, a new high-performance, open-source* subset of Atari environments re-designed for the experimental evaluation of deep RL. We show that TOYBOX enables a wide range of experiments and analyses that are impossible in other environments. *https://kdl-umass.github.io/Toybox/