Can LLMs Fix Issues with Reasoning Models? Towards More Likely Models for AI Planning

TL;DR

This paper investigates the use of large language models (LLMs) for model space editing in AI planning, comparing their performance to traditional combinatorial search methods and exploring their potential to improve reasoning in planning tasks.

Contribution

It is the first study to evaluate LLMs for model space reasoning in AI planning and compares their effectiveness with established combinatorial search techniques.

Findings

LLMs show promising performance in model space reasoning tasks.

LLMs can complement combinatorial search methods in planning.

Further research is encouraged to enhance LLMs for planning applications.

Abstract

This is the first work to look at the application of large language models (LLMs) for the purpose of model space edits in automated planning tasks. To set the stage for this union, we explore two different flavors of model space problems that have been studied in the AI planning literature and explore the effect of an LLM on those tasks. We empirically demonstrate how the performance of an LLM contrasts with combinatorial search (CS) -- an approach that has been traditionally used to solve model space tasks in planning, both with the LLM in the role of a standalone model space reasoner as well as in the role of a statistical signal in concert with the CS approach as part of a two-stage process. Our experiments show promising results suggesting further forays of LLMs into the exciting world of model space reasoning for planning tasks in the future.