Bigger, Better, Faster: Human-level Atari with human-level efficiency

TL;DR

This paper presents BBF, a value-based reinforcement learning agent that achieves super-human performance in Atari 100K with enhanced neural network scaling and design choices, setting new standards for sample efficiency.

Contribution

Introduction of BBF, a scalable, sample-efficient RL agent that surpasses human-level performance in Atari 100K benchmarks, with extensive analysis of design choices.

Findings

BBF achieves super-human performance in Atari 100K.

Scaling neural networks improves sample efficiency.

Design choices significantly impact RL performance.

Abstract

We introduce a value-based RL agent, which we call BBF, that achieves super-human performance in the Atari 100K benchmark. BBF relies on scaling the neural networks used for value estimation, as well as a number of other design choices that enable this scaling in a sample-efficient manner. We conduct extensive analyses of these design choices and provide insights for future work. We end with a discussion about updating the goalposts for sample-efficient RL research on the ALE. We make our code and data publicly available at https://github.com/google-research/google-research/tree/master/bigger_better_faster.