Solving Hard AI Planning Instances Using Curriculum-Driven Deep Reinforcement Learning

TL;DR

This paper introduces a curriculum-driven deep reinforcement learning approach that successfully solves hard Sokoban planning instances within a day, outperforming existing solvers and automatically discovering domain structure.

Contribution

It presents the first deep RL method capable of solving challenging Sokoban instances within a day, automatically uncovering domain structure without handcrafted pruning.

Findings

Deep RL solves previously unsolvable Sokoban instances.

Curriculum-driven training accelerates learning of complex planning tasks.

Automatic domain structure discovery enhances planning efficiency.

Abstract

Despite significant progress in general AI planning, certain domains remain out of reach of current AI planning systems. Sokoban is a PSPACE-complete planning task and represents one of the hardest domains for current AI planners. Even domain-specific specialized search methods fail quickly due to the exponential search complexity on hard instances. Our approach based on deep reinforcement learning augmented with a curriculum-driven method is the first one to solve hard instances within one day of training while other modern solvers cannot solve these instances within any reasonable time limit. In contrast to prior efforts, which use carefully handcrafted pruning techniques, our approach automatically uncovers domain structure. Our results reveal that deep RL provides a promising framework for solving previously unsolved AI planning problems, provided a proper training curriculum can be devised.