Neural architecture impact on identifying temporally extended Reinforcement Learning tasks

TL;DR

This paper explores attention-based neural architectures, including Vision Transformers, for reinforcement learning in Atari games, enhancing interpretability and efficiency compared to previous models.

Contribution

Introduces attention-based models, including a Vision Transformer architecture, for RL that improve interpretability and training efficiency in Atari environments.

Findings

Attention models enable visualization of agent perception.

Vision Transformer-based RL models train faster and use fewer resources.

Models perform well on Atari-2600 games.

Abstract

Inspired by recent developments in attention models for image classification and natural language processing, we present various Attention based architectures in reinforcement learning (RL) domain, capable of performing well on OpenAI Gym Atari-2600 game suite. In spite of the recent success of Deep Reinforcement learning techniques in various fields like robotics, gaming and healthcare, they suffer from a major drawback that neural networks are difficult to interpret. We try to get around this problem with the help of Attention based models. In Attention based models, extracting and overlaying of attention map onto images allows for direct observation of information used by agent to select actions and easier interpretation of logic behind the chosen actions. Our models in addition to playing well on gym-Atari environments, also provide insights on how agent perceives its environment. In addition, motivated by recent developments in attention based video-classification models using Vision Transformer, we come up with an architecture based on Vision Transformer, for image-based RL domain too. Compared to previous works in Vision Transformer, our model is faster to train and requires fewer computational resources. 3