Playing Doom with SLAM-Augmented Deep Reinforcement Learning

TL;DR

This paper introduces a novel approach that enhances deep reinforcement learning in complex 3D environments by integrating semantic abstractions like object detection and scene reconstruction, leading to improved policy learning in Doom.

Contribution

The paper presents a method to incorporate semantic concepts and scene structure into deep RL, addressing challenges in 3D environments and improving policy effectiveness.

Findings

Augmented DQN outperforms standard DQN in Doom.

Semantic scene understanding improves policy learning.

Method effectively handles partial observability and exploration challenges.

Abstract

A number of recent approaches to policy learning in 2D game domains have been successful going directly from raw input images to actions. However when employed in complex 3D environments, they typically suffer from challenges related to partial observability, combinatorial exploration spaces, path planning, and a scarcity of rewarding scenarios. Inspired from prior work in human cognition that indicates how humans employ a variety of semantic concepts and abstractions (object categories, localisation, etc.) to reason about the world, we build an agent-model that incorporates such abstractions into its policy-learning framework. We augment the raw image input to a Deep Q-Learning Network (DQN), by adding details of objects and structural elements encountered, along with the agent's localisation. The different components are automatically extracted and composed into a topological representation using on-the-fly object detection and 3D-scene reconstruction.We evaluate the efficacy of our approach in Doom, a 3D first-person combat game that exhibits a number of challenges discussed, and show that our augmented framework consistently learns better, more effective policies.