Planning in the Dark: LLM-Symbolic Planning Pipeline without Experts

TL;DR

This paper introduces an automated LLM-symbolic planning pipeline that constructs multiple action schemas, validates and ranks them without expert intervention, improving planning robustness and accessibility.

Contribution

It presents a novel method to generate and validate multiple candidate schemas automatically, eliminating the need for expert intervention in LLM-symbolic planning.

Findings

Outperforms direct LLM planning in experiments

Automates schema validation and ranking

Enables fully automated end-to-end planning

Abstract

Large Language Models (LLMs) have shown promise in solving natural language-described planning tasks, but their direct use often leads to inconsistent reasoning and hallucination. While hybrid LLM-symbolic planning pipelines have emerged as a more robust alternative, they typically require extensive expert intervention to refine and validate generated action schemas. It not only limits scalability but also introduces a potential for biased interpretation, as a single expert's interpretation of ambiguous natural language descriptions might not align with the user's actual intent. To address this, we propose a novel approach that constructs an action schema library to generate multiple candidates, accounting for the diverse possible interpretations of natural language descriptions. We further introduce a semantic validation and ranking module that automatically filter and rank the generated schemas and plans without expert-in-the-loop. The experiments showed our pipeline maintains superiority in planning over the direct LLM planning approach. These findings demonstrate the feasibility of a fully automated end-to-end LLM-symbolic planner that requires no expert intervention, opening up the possibility for a broader audience to engage with AI planning with less prerequisite of domain expertise.