Value Iteration Networks

TL;DR

The paper presents the value iteration network (VIN), a neural network with an embedded planning module that learns to perform planning-based reasoning, improving generalization in path-planning and search tasks.

Contribution

It introduces a differentiable approximation of value iteration as a neural network module, enabling end-to-end training for planning tasks.

Findings

VIN policies outperform baselines in path-planning tasks.

VIN generalizes better to unseen domains.

The approach integrates planning into neural networks effectively.

Abstract

We introduce the value iteration network (VIN): a fully differentiable neural network with a `planning module' embedded within. VINs can learn to plan, and are suitable for predicting outcomes that involve planning-based reasoning, such as policies for reinforcement learning. Key to our approach is a novel differentiable approximation of the value-iteration algorithm, which can be represented as a convolutional neural network, and trained end-to-end using standard backpropagation. We evaluate VIN based policies on discrete and continuous path-planning domains, and on a natural-language based search task. We show that by learning an explicit planning computation, VIN policies generalize better to new, unseen domains.