A Roadmap to Guide the Integration of LLMs in Hierarchical Planning

TL;DR

This paper proposes a roadmap and taxonomy for integrating Large Language Models into Hierarchical Planning, introduces a benchmark dataset, and provides initial performance results to guide future research in this emerging area.

Contribution

It introduces a novel taxonomy for LLM integration in Hierarchical Planning and provides a benchmark dataset with baseline results for future evaluation.

Findings

LLM-based planner achieved 3% correct plans.

No correct hierarchical decompositions were produced by the LLM planner.

The baseline results highlight the challenges and potential for future improvements.

Abstract

Recent advances in Large Language Models (LLMs) are fostering their integration into several reasoning-related fields, including Automated Planning (AP). However, their integration into Hierarchical Planning (HP), a subfield of AP that leverages hierarchical knowledge to enhance planning performance, remains largely unexplored. In this preliminary work, we propose a roadmap to address this gap and harness the potential of LLMs for HP. To this end, we present a taxonomy of integration methods, exploring how LLMs can be utilized within the HP life cycle. Additionally, we provide a benchmark with a standardized dataset for evaluating the performance of future LLM-based HP approaches, and present initial results for a state-of-the-art HP planner and LLM planner. As expected, the latter exhibits limited performance (3\% correct plans, and none with a correct hierarchical decomposition) but serves as a valuable baseline for future approaches.