A Differentiable Loss Function for Learning Heuristics in A*

TL;DR

This paper introduces a new differentiable loss function for training neural heuristics in A* search, focusing on reducing unnecessary state expansions and improving planning efficiency in maze domains.

Contribution

It proposes the L* loss, which better aligns neural network training with A* search efficiency, outperforming traditional square root loss methods.

Findings

L* loss reduces expanded states by approximately 50%

Improves the fraction of solved problems in maze planning

Enhances the quality of generated plans

Abstract

Optimization of heuristic functions for the A* algorithm, realized by deep neural networks, is usually done by minimizing square root loss of estimate of the cost to goal values. This paper argues that this does not necessarily lead to a faster search of A* algorithm since its execution relies on relative values instead of absolute ones. As a mitigation, we propose a L* loss, which upper-bounds the number of excessively expanded states inside the A* search. The L* loss, when used in the optimization of state-of-the-art deep neural networks for automated planning in maze domains like Sokoban and maze with teleports, significantly improves the fraction of solved problems, the quality of founded plans, and reduces the number of expanded states to approximately 50%