Compositional Coordination for Multi-Robot Teams with Large Language Models

TL;DR

LAN2CB leverages large language models to convert natural language mission descriptions into executable multi-robot control code, simplifying coordination and reducing manual engineering effort.

Contribution

The paper introduces LAN2CB, a novel framework that automates multi-robot coordination from natural language using LLMs, behavior trees, and a new dataset for benchmarking.

Findings

Enables robust multi-robot coordination from natural language

Reduces manual engineering effort significantly

Supports broad generalization across diverse missions

Abstract

Multi-robot coordination has traditionally relied on a mission-specific and expert-driven pipeline, where natural language mission descriptions are manually translated by domain experts into mathematical formulation, algorithm design, and executable code. This conventional process is labor-intensive, inaccessible to non-experts, and inflexible to changes in mission requirements. Here, we propose LAN2CB (Language to Collective Behavior), a novel framework that leverages large language models (LLMs) to streamline and generalize the multi-robot coordination pipeline. LAN2CB transforms natural language (NL) mission descriptions into executable Python code for multi-robot systems through two core modules: (1) Mission Analysis, which parses mission descriptions into behavior trees, and (2) Code Generation, which leverages the behavior tree and a structured knowledge base to generate robot control code. We further introduce a dataset of natural language mission descriptions to support development and benchmarking. Experiments in both simulation and real-world environments demonstrate that LAN2CB enables robust and flexible multi-robot coordination from natural language, significantly reducing manual engineering effort and supporting broad generalization across diverse mission types. Website: https://sites.google.com/view/lan-cb