Understanding the Capabilities of Large Language Models for Automated Planning

TL;DR

This paper investigates the potential of large language models to perform automated planning, examining their capabilities, optimal training methods, and generalization abilities in generating plans for complex tasks.

Contribution

It provides a comprehensive analysis of how LLMs can be applied to automated planning, comparing fine-tuning and prompting strategies, and identifying effective pre-training data for plan generation.

Findings

LLMs can generate plans with reasonable accuracy.

Prompting often outperforms fine-tuning in plan generation.

Pre-training data significantly influences planning performance.

Abstract

Automated planning is concerned with developing efficient algorithms to generate plans or sequences of actions to achieve a specific goal in a given environment. Emerging Large Language Models (LLMs) can answer questions, write high-quality programming code, and predict protein folding, showcasing their versatility in solving various tasks beyond language-based problems. In this paper, we aim to explore how LLMs can also be used for automated planning. To do so, we seek to answer four key questions. Firstly, we want to understand the extent to which LLMs can be used for plan generation. Secondly, we aim to identify which pre-training data is most effective in facilitating plan generation. Thirdly, we investigate whether fine-tuning or prompting is a more effective approach for plan generation. Finally, we explore whether LLMs are capable of plan generalization. By answering these questions, the study seeks to shed light on the capabilities of LLMs in solving complex planning problems and provide insights into the most effective approaches for using LLMs in this context.