Recurrent Environment Simulators

TL;DR

This paper introduces recurrent neural network-based environment simulators capable of long-term, coherent predictions from high-dimensional pixel data, enhancing planning and exploration across diverse environments.

Contribution

It presents a novel recurrent model that efficiently predicts environment dynamics over many time steps without generating high-dimensional images at each step.

Findings

Improved long-term prediction accuracy.

Enhanced exploration capabilities in various environments.

Effective adaptation to diverse complex environments.

Abstract

Models that can simulate how environments change in response to actions can be used by agents to plan and act efficiently. We improve on previous environment simulators from high-dimensional pixel observations by introducing recurrent neural networks that are able to make temporally and spatially coherent predictions for hundreds of time-steps into the future. We present an in-depth analysis of the factors affecting performance, providing the most extensive attempt to advance the understanding of the properties of these models. We address the issue of computationally inefficiency with a model that does not need to generate a high-dimensional image at each time-step. We show that our approach can be used to improve exploration and is adaptable to many diverse environments, namely 10 Atari games, a 3D car racing environment, and complex 3D mazes.