Generating consistent PDDL domains with Large Language Models

TL;DR

This paper introduces a method to improve the quality of PDDL domain models generated by Large Language Models by incorporating automated consistency checks, reducing human correction efforts.

Contribution

It presents a novel automated consistency checking approach integrated into the generation process of LLMs for PDDL domains, enhancing model reliability.

Findings

Consistency checking improves PDDL model quality

Error detection reduces human correction efforts

Effective on multiple classical and custom domains

Abstract

Large Language Models (LLMs) are capable of transforming natural language domain descriptions into plausibly looking PDDL markup. However, ensuring that actions are consistent within domains still remains a challenging task. In this paper we present a novel concept to significantly improve the quality of LLM-generated PDDL models by performing automated consistency checking during the generation process. Although the proposed consistency checking strategies still can't guarantee absolute correctness of generated models, they can serve as valuable source of feedback reducing the amount of correction efforts expected from a human in the loop. We demonstrate the capabilities of our error detection approach on a number of classical and custom planning domains (logistics, gripper, tyreworld, household, pizza).