Investigating Simple Object Representations in Model-Free Deep Reinforcement Learning

TL;DR

This paper demonstrates that augmenting deep reinforcement learning agents with simple object representations significantly improves their performance and generalization in Atari games, addressing a key cognitive gap.

Contribution

It introduces the integration of simple object representations into model-free deep RL algorithms and analyzes their impact on performance and generalization.

Findings

Object representations boost Atari Frostbite performance.

Different object types contribute variably to learning.

Representations improve generalization to new situations.

Abstract

We explore the benefits of augmenting state-of-the-art model-free deep reinforcement algorithms with simple object representations. Following the Frostbite challenge posited by Lake et al. (2017), we identify object representations as a critical cognitive capacity lacking from current reinforcement learning agents. We discover that providing the Rainbow model (Hessel et al.,2018) with simple, feature-engineered object representations substantially boosts its performance on the Frostbite game from Atari 2600. We then analyze the relative contributions of the representations of different types of objects, identify environment states where these representations are most impactful, and examine how these representations aid in generalizing to novel situations.