Heuristic Search Planning with Deep Neural Networks using Imitation, Attention and Curriculum Learning

TL;DR

This paper introduces a neural network-based heuristic for planning that uses imitation, attention, and curriculum learning to significantly outperform existing methods in complex grid domains.

Contribution

It presents a novel neural network model with attention and curriculum learning to improve heuristic quality for complex planning tasks.

Findings

Outperforms classical heuristics in grid PDDL domains

Uses imitation learning to relate distant state space parts

Curriculum learning enhances performance on harder problems

Abstract

Learning a well-informed heuristic function for hard task planning domains is an elusive problem. Although there are known neural network architectures to represent such heuristic knowledge, it is not obvious what concrete information is learned and whether techniques aimed at understanding the structure help in improving the quality of the heuristics. This paper presents a network model to learn a heuristic capable of relating distant parts of the state space via optimal plan imitation using the attention mechanism, which drastically improves the learning of a good heuristic function. To counter the limitation of the method in the creation of problems of increasing difficulty, we demonstrate the use of curriculum learning, where newly solved problem instances are added to the training set, which, in turn, helps to solve problems of higher complexities and far exceeds the performances of all existing baselines including classical planning heuristics. We demonstrate its effectiveness for grid-type PDDL domains.