Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physics

TL;DR

The paper introduces Schema Networks, an object-oriented generative physics model that learns environment dynamics from data, enabling efficient zero-shot transfer and robust generalization in reinforcement learning tasks.

Contribution

It presents a novel Schema Network architecture that models causal physics, improving transfer and generalization over existing deep learning methods.

Findings

Schema Networks outperform A3C and Progressive Networks in transfer tasks.

They demonstrate faster learning and better zero-shot generalization.

The approach effectively learns environment dynamics from limited data.

Abstract

The recent adaptation of deep neural network-based methods to reinforcement learning and planning domains has yielded remarkable progress on individual tasks. Nonetheless, progress on task-to-task transfer remains limited. In pursuit of efficient and robust generalization, we introduce the Schema Network, an object-oriented generative physics simulator capable of disentangling multiple causes of events and reasoning backward through causes to achieve goals. The richly structured architecture of the Schema Network can learn the dynamics of an environment directly from data. We compare Schema Networks with Asynchronous Advantage Actor-Critic and Progressive Networks on a suite of Breakout variations, reporting results on training efficiency and zero-shot generalization, consistently demonstrating faster, more robust learning and better transfer. We argue that generalizing from limited data and learning causal relationships are essential abilities on the path toward generally intelligent systems.