Using Language Models as Closed-Loop High-Level Planners for Robotics Applications: A Brief Overview and Benchmarks

TL;DR

This paper investigates how to reliably integrate large language models as high-level planners in robotics, focusing on control horizon and warm-starting strategies to enhance performance and robustness.

Contribution

It provides empirical insights and practical recommendations for improving language model-based planning in robotic systems through controlled experiments.

Findings

Control horizon and warm-starting significantly affect planning performance.

Designed experiments yield actionable insights for robust LLM-based planning.

Implementation details and experiments are publicly available.

Abstract

Large Language Models (LLMs) and Vision Language Models (VLMs) have become popular tools for embodied high-level planning. However, their deployment in black-box settings often leads to unpredictable or costly errors. To harness their capabilities more reliably in robotic systems, we empirically investigate practical strategies for integrating language models as closed-loop planners. Concretely, we study how the control horizon and warm-starting impact the performance of language model-based planners. We design and conduct controlled experiments to extract actionable insights, providing recommendations that can help improve the performance and robustness of language model-based embodied planning. The full implementation and experiments are available on the project website