Agent57: Outperforming the Atari Human Benchmark

TL;DR

Agent57 is a deep reinforcement learning agent that outperforms the average human benchmark across all 57 Atari games by using adaptive policy selection and a novel neural network architecture.

Contribution

It introduces a new deep RL agent with adaptive policy switching and a stable architecture, surpassing human performance on all Atari benchmark games.

Findings

Outperforms human benchmark on all 57 Atari games

Uses adaptive policy selection for exploration and exploitation

Achieves more consistent and stable learning results

Abstract

Atari games have been a long-standing benchmark in the reinforcement learning (RL) community for the past decade. This benchmark was proposed to test general competency of RL algorithms. Previous work has achieved good average performance by doing outstandingly well on many games of the set, but very poorly in several of the most challenging games. We propose Agent57, the first deep RL agent that outperforms the standard human benchmark on all 57 Atari games. To achieve this result, we train a neural network which parameterizes a family of policies ranging from very exploratory to purely exploitative. We propose an adaptive mechanism to choose which policy to prioritize throughout the training process. Additionally, we utilize a novel parameterization of the architecture that allows for more consistent and stable learning.