Imagination-Augmented Agents for Deep Reinforcement Learning

TL;DR

Imagination-Augmented Agents (I2As) integrate learned environment models with deep policy networks, enhancing data efficiency, performance, and robustness in deep reinforcement learning by constructing implicit plans from model predictions.

Contribution

The paper introduces I2As, a novel architecture that combines model-based and model-free reinforcement learning by interpreting environment model predictions as context for policy networks.

Findings

I2As outperform baselines in data efficiency.

I2As demonstrate improved robustness to model errors.

I2As achieve higher performance in reinforcement learning tasks.

Abstract

We introduce Imagination-Augmented Agents (I2As), a novel architecture for deep reinforcement learning combining model-free and model-based aspects. In contrast to most existing model-based reinforcement learning and planning methods, which prescribe how a model should be used to arrive at a policy, I2As learn to interpret predictions from a learned environment model to construct implicit plans in arbitrary ways, by using the predictions as additional context in deep policy networks. I2As show improved data efficiency, performance, and robustness to model misspecification compared to several baselines.