Prime the search: Using large language models for guiding geometric task and motion planning by warm-starting tree search

TL;DR

This paper introduces a hybrid approach combining Large Language Models and Monte Carlo Tree Search to efficiently guide geometric task and motion planning, leveraging common sense reasoning to improve planning performance.

Contribution

It proposes a novel method that uses LLMs to warm-start MCTS in G-TAMP problems, reducing computational costs and enhancing planning efficiency.

Findings

Outperforms previous LLM-based planners

Achieves better success rates on six G-TAMP problems

Reduces computational costs compared to prior methods

Abstract

The problem of relocating a set of objects to designated areas amidst movable obstacles can be framed as a Geometric Task and Motion Planning (G-TAMP) problem, a subclass of task and motion planning (TAMP). Traditional approaches to G-TAMP have relied either on domain-independent heuristics or on learning from planning experience to guide the search, both of which typically demand significant computational resources or data. In contrast, humans often use common sense to intuitively decide which objects to manipulate in G-TAMP problems. Inspired by this, we propose leveraging Large Language Models (LLMs), which have common sense knowledge acquired from internet-scale data, to guide task planning in G-TAMP problems. To enable LLMs to perform geometric reasoning, we design a predicate-based prompt that encodes geometric information derived from a motion planning algorithm. We then query the LLM to generate a task plan, which is then used to search for a feasible set of continuous parameters. Since LLMs are prone to mistakes, instead of committing to LLM's outputs, we extend Monte Carlo Tree Search (MCTS) to a hybrid action space and use the LLM to guide the search. Unlike the previous approach that calls an LLM at every node and incurs high computational costs, we use it to warm-start the MCTS with the nodes explored in completing the LLM's task plan. On six different G-TAMP problems, we show our method outperforms previous LLM planners and pure search algorithms. Code can be found at: https://github.com/iMSquared/prime-the-search