Model Space Reasoning as Search in Feedback Space for Planning Domain Generation

TL;DR

This paper proposes a search-based framework using feedback mechanisms to improve the generation of planning domains from natural language descriptions, leveraging heuristic search and symbolic validation.

Contribution

It introduces a novel agentic language model feedback framework that incorporates symbolic feedback to enhance domain generation quality from natural language.

Findings

Feedback mechanisms improve domain quality

Heuristic search effectively optimizes model outputs

Symbolic validation guides better domain generation

Abstract

The generation of planning domains from natural language descriptions remains an open problem even with the advent of large language models and reasoning models. Recent work suggests that while LLMs have the ability to assist with domain generation, they are still far from producing high quality domains that can be deployed in practice. To this end, we investigate the ability of an agentic language model feedback framework to generate planning domains from natural language descriptions that have been augmented with a minimal amount of symbolic information. In particular, we evaluate the quality of the generated domains under various forms of symbolic feedback, including landmarks, and output from the VAL plan validator. Using these feedback mechanisms, we experiment using heuristic search over model space to optimize domain quality.