Unifying Inference-Time Planning Language Generation

TL;DR

This paper unifies various inference-time planning language generation methods using a common framework based on intermediate representations, systematically evaluating multiple pipelines to improve robustness and performance.

Contribution

It introduces a unifying framework for inference-time LLM-based planning language generation and evaluates diverse pipelines, including novel high-resource intermediate languages.

Findings

Unified framework improves consistency across methods

Intermediate representations enhance robustness against complexity

Novel pipelines with high-resource languages show promising results

Abstract

A line of work in planning uses LLM not to generate a plan, but to generate a formal representation in some planning language, which can be input into a symbolic solver to deterministically find a plan. While showing improved trust and promising performance, dozens of recent publications have proposed scattered methods on a variety of benchmarks under different experimental settings. We attempt to unify the inference-time LLM-as-formalizer methodology for classical planning by proposing a unifying framework based on intermediate representations. We thus systematically evaluate more than a dozen pipelines that subsume most existing work, while proposing novel ones that involve syntactically similar but high resource intermediate languages (such as a Python wrapper of PDDL). We provide recipes for planning language generation pipelines, draw a series of conclusions showing the efficacy of their various components, and evidence their robustness against problem complexity.