Control of Memory, Active Perception, and Action in Minecraft

TL;DR

This paper introduces new reinforcement learning tasks in Minecraft to evaluate and compare deep RL architectures, highlighting the importance of memory and active perception for better generalization in complex, partially observable environments.

Contribution

The paper presents novel RL tasks in Minecraft and proposes memory-based DRL architectures that outperform existing methods in generalization to unseen environments.

Findings

Memory-based architectures generalize better to new environments.

New tasks effectively challenge existing DRL methods.

Active perception improves performance in complex visual tasks.

Abstract

In this paper, we introduce a new set of reinforcement learning (RL) tasks in Minecraft (a flexible 3D world). We then use these tasks to systematically compare and contrast existing deep reinforcement learning (DRL) architectures with our new memory-based DRL architectures. These tasks are designed to emphasize, in a controllable manner, issues that pose challenges for RL methods including partial observability (due to first-person visual observations), delayed rewards, high-dimensional visual observations, and the need to use active perception in a correct manner so as to perform well in the tasks. While these tasks are conceptually simple to describe, by virtue of having all of these challenges simultaneously they are difficult for current DRL architectures. Additionally, we evaluate the generalization performance of the architectures on environments not used during training. The experimental results show that our new architectures generalize to unseen environments better than existing DRL architectures.