Pixel to policy: DQN Encoders for within & cross-game reinforcement learning

TL;DR

This paper investigates transfer learning in reinforcement learning by using DQN encoders trained on multiple games, achieving high performance with fewer episodes and exploring cross-game transfer for universal game agents.

Contribution

It introduces a transfer learning approach with DQN encoders trained across multiple environments, demonstrating improved efficiency and performance over training from scratch.

Findings

DQN achieves a mean episode reward of 46.16, surpassing human performance with only 20k episodes.

The model attains mean rewards of 533.42 and 402.17 on Assault and Space Invaders, respectively.

Transfer learning with pre-trained encoders enhances learning efficiency across different games.

Abstract

Reinforcement Learning can be applied to various tasks, and environments. Many of these environments have a similar shared structure, which can be exploited to improve RL performance on other tasks. Transfer learning can be used to take advantage of this shared structure, by learning policies that are transferable across different tasks and environments and can lead to more efficient learning as well as improved performance on a wide range of tasks. This work explores as well as compares the performance between RL models being trained from the scratch and on different approaches of transfer learning. Additionally, the study explores the performance of a model trained on multiple game environments, with the goal of developing a universal game-playing agent as well as transfer learning a pre-trained encoder using DQN, and training it on the same game or a different game. Our DQN model achieves a mean episode reward of 46.16 which even beats the human-level performance with merely 20k episodes which is significantly lower than deepmind's 1M episodes. The achieved mean rewards of 533.42 and 402.17 on the Assault and Space Invader environments respectively, represent noteworthy performance on these challenging environments.