Probabilistically Correct Language-based Multi-Robot Planning using Conformal Prediction

TL;DR

This paper introduces S-ATLAS, a distributed multi-robot planning system that uses conformal prediction to provide probabilistic guarantees on task success rates while minimizing help requests, improving efficiency and reliability.

Contribution

The paper presents a novel LLM-based distributed planner with conformal prediction for uncertainty quantification, enabling guaranteed success rates in multi-robot language-instructed tasks.

Findings

Achieves user-specified success rates assuming plan execution

Reduces help requests compared to existing methods

More scalable and efficient with larger robot teams

Abstract

This paper addresses task planning problems for language-instructed robot teams. Tasks are expressed in natural language (NL), requiring the robots to apply their capabilities at various locations and semantic objects. Several recent works have addressed similar planning problems by leveraging pre-trained Large Language Models (LLMs) to design effective multi-robot plans. However, these approaches lack performance guarantees. To address this challenge, we introduce a new distributed LLM-based planner, called S-ATLAS for Safe plAnning for Teams of Language-instructed AgentS, that is capable of achieving user-defined mission success rates. This is accomplished by leveraging conformal prediction (CP), a distribution-free uncertainty quantification tool in black-box models. CP allows the proposed multi-robot planner to reason about its inherent uncertainty in a distributed fashion, enabling robots to make individual decisions when they are sufficiently certain and seek help otherwise. We show, both theoretically and empirically, that the proposed planner can achieve user-specified task success rates, assuming successful plan execution, while minimizing the overall number of help requests. We provide comparative experiments against related works showing that our method is significantly more computational efficient and achieves lower help rates. The advantage of our algorithm over baselines becomes more pronounced with increasing robot team size.