Subgoal Search For Complex Reasoning Tasks

TL;DR

This paper introduces kSubS, a subgoal search method using learned subgoal generation to improve planning efficiency in complex reasoning tasks, demonstrating state-of-the-art results on benchmarks like INT.

Contribution

The paper presents a novel transformer-based subgoal generator integrated with classical search, enabling efficient exploration in complex reasoning domains.

Findings

kSubS achieves state-of-the-art results on the INT benchmark.

The method is effective on puzzle games like Sokoban and Rubik's Cube.

Subgoal generation significantly reduces search complexity.

Abstract

Humans excel in solving complex reasoning tasks through a mental process of moving from one idea to a related one. Inspired by this, we propose Subgoal Search (kSubS) method. Its key component is a learned subgoal generator that produces a diversity of subgoals that are both achievable and closer to the solution. Using subgoals reduces the search space and induces a high-level search graph suitable for efficient planning. In this paper, we implement kSubS using a transformer-based subgoal module coupled with the classical best-first search framework. We show that a simple approach of generating $k$-th step ahead subgoals is surprisingly efficient on three challenging domains: two popular puzzle games, Sokoban and the Rubik's Cube, and an inequality proving benchmark INT. kSubS achieves strong results including state-of-the-art on INT within a modest computational budget.