Deep Reinforcement Learning From Raw Pixels in Doom

TL;DR

This paper explores applying deep reinforcement learning to a complex 3D Doom environment, analyzing challenges, training algorithms, and providing insights into learned behaviors despite limited success.

Contribution

It introduces a new Doom task for RL agents and evaluates DQN and LSTM-A3C algorithms, highlighting challenges and initial behaviors in this complex setting.

Findings

Both algorithms learned sensible policies

High scores were not achieved with current training

Insights into learned behaviors were provided

Abstract

Using current reinforcement learning methods, it has recently become possible to learn to play unknown 3D games from raw pixels. In this work, we study the challenges that arise in such complex environments, and summarize current methods to approach these. We choose a task within the Doom game, that has not been approached yet. The goal for the agent is to fight enemies in a 3D world consisting of five rooms. We train the DQN and LSTM-A3C algorithms on this task. Results show that both algorithms learn sensible policies, but fail to achieve high scores given the amount of training. We provide insights into the learned behavior, which can serve as a valuable starting point for further research in the Doom domain.