Visualizing and Understanding Atari Agents

TL;DR

This paper uses saliency maps to interpret Atari reinforcement learning agents, revealing their decision-making strategies, attention focus, and learning evolution, thereby enhancing understanding of deep RL behaviors.

Contribution

Introduces a novel saliency map method for analyzing Atari agents, enabling insights into their attention, decision reasons, and learning dynamics.

Findings

Saliency maps reveal what agents attend to during decision-making

Agents sometimes make decisions for the wrong reasons

Saliency visualization improves human understanding of RL agents

Abstract

While deep reinforcement learning (deep RL) agents are effective at maximizing rewards, it is often unclear what strategies they use to do so. In this paper, we take a step toward explaining deep RL agents through a case study using Atari 2600 environments. In particular, we focus on using saliency maps to understand how an agent learns and executes a policy. We introduce a method for generating useful saliency maps and use it to show 1) what strong agents attend to, 2) whether agents are making decisions for the right or wrong reasons, and 3) how agents evolve during learning. We also test our method on non-expert human subjects and find that it improves their ability to reason about these agents. Overall, our results show that saliency information can provide significant insight into an RL agent's decisions and learning behavior.