Towards a General Framework for HTN Modeling with LLMs

TL;DR

This paper introduces L2HP, a framework extending L2P for hierarchical planning model generation using LLMs, and compares their capabilities in automated planning and hierarchical planning, revealing significant challenges in HP model validity.

Contribution

The paper presents L2HP, a novel extension supporting hierarchical planning model generation with LLMs, and provides an empirical comparison highlighting challenges in HP model quality.

Findings

Parsing success is around 36% for both AP and HP.

Syntactic validity is significantly lower in HP (1%) compared to AP (20%).

Hierarchical planning models pose unique challenges for LLMs.

Abstract

The use of Large Language Models (LLMs) for generating Automated Planning (AP) models has been widely explored; however, their application to Hierarchical Planning (HP) is still far from reaching the level of sophistication observed in non-hierarchical architectures. In this work, we try to address this gap. We present two main contributions. First, we propose L2HP, an extension of L2P (a library to LLM-driven PDDL models generation) that support HP model generation and follows a design philosophy of generality and extensibility. Second, we apply our framework to perform experiments where we compare the modeling capabilities of LLMs for AP and HP. On the PlanBench dataset, results show that parsing success is limited but comparable in both settings (around 36\%), while syntactic validity is substantially lower in the hierarchical case (1\% vs. 20\% of instances). These findings underscore the unique challenges HP presents for LLMs, highlighting the need for further research to improve the quality of generated HP models.