World Models

TL;DR

This paper introduces a generative world model for reinforcement learning environments that enables training agents within their own simulated 'dreams', leading to efficient policy learning and transfer.

Contribution

It presents a novel approach where agents learn and plan within a learned environment model, allowing for efficient training and policy transfer.

Findings

Agents can be trained entirely inside their hallucinated environment.

The world model enables quick unsupervised learning of environment representations.

Policies trained in the model can be transferred to real environments.

Abstract

We explore building generative neural network models of popular reinforcement learning environments. Our world model can be trained quickly in an unsupervised manner to learn a compressed spatial and temporal representation of the environment. By using features extracted from the world model as inputs to an agent, we can train a very compact and simple policy that can solve the required task. We can even train our agent entirely inside of its own hallucinated dream generated by its world model, and transfer this policy back into the actual environment. An interactive version of this paper is available at https://worldmodels.github.io/