Large Language Models as Commonsense Knowledge for Large-Scale Task Planning

TL;DR

This paper introduces LLM-MCTS, a novel approach that combines large language models as a world model and policy to enhance large-scale task planning, significantly outperforming previous methods on complex tasks.

Contribution

The paper presents a new method integrating LLMs with Monte Carlo Tree Search, leveraging LLMs as both a world model and policy to improve planning efficiency and effectiveness.

Findings

LLM-MCTS outperforms standalone MCTS and LLM-induced policies on complex tasks.

Using LLMs as a world model provides a strong prior for effective reasoning.

Minimum description length guides when LLM-based world models are preferable.

Abstract

Large-scale task planning is a major challenge. Recent work exploits large language models (LLMs) directly as a policy and shows surprisingly interesting results. This paper shows that LLMs provide a commonsense model of the world in addition to a policy that acts on it. The world model and the policy can be combined in a search algorithm, such as Monte Carlo Tree Search (MCTS), to scale up task planning. In our new LLM-MCTS algorithm, the LLM-induced world model provides a commonsense prior belief for MCTS to achieve effective reasoning; the LLM-induced policy acts as a heuristic to guide the search, vastly improving search efficiency. Experiments show that LLM-MCTS outperforms both MCTS alone and policies induced by LLMs (GPT2 and GPT3.5) by a wide margin, for complex, novel tasks. Further experiments and analyses on multiple tasks -- multiplication, multi-hop travel planning, object rearrangement -- suggest minimum description length (MDL) as a general guiding principle: if the description length of the world model is substantially smaller than that of the policy, using LLM as a world model for model-based planning is likely better than using LLM solely as a policy.