Deep Attention Recurrent Q-Network

TL;DR

This paper introduces DARQN, an extension of Deep Q-Networks that incorporates attention mechanisms, improving performance on Atari games and enabling visualization of the agent’s focus during decision-making.

Contribution

The paper presents a novel Deep Attention Recurrent Q-Network that integrates soft and hard attention mechanisms into DQN, enhancing performance and interpretability in reinforcement learning tasks.

Findings

DARQN outperforms DQN on multiple Atari games.

Attention mechanisms enable real-time visualization of focus regions.

The approach improves learning efficiency and decision interpretability.

Abstract

A deep learning approach to reinforcement learning led to a general learner able to train on visual input to play a variety of arcade games at the human and superhuman levels. Its creators at the Google DeepMind's team called the approach: Deep Q-Network (DQN). We present an extension of DQN by "soft" and "hard" attention mechanisms. Tests of the proposed Deep Attention Recurrent Q-Network (DARQN) algorithm on multiple Atari 2600 games show level of performance superior to that of DQN. Moreover, built-in attention mechanisms allow a direct online monitoring of the training process by highlighting the regions of the game screen the agent is focusing on when making decisions.