DeepFleet: Multi-Agent Foundation Models for Mobile Robots

TL;DR

DeepFleet introduces multi-agent foundation models trained on extensive warehouse robot data, exploring various architectures for improved coordination and planning in large-scale robot fleets.

Contribution

The paper presents four novel multi-agent foundation model architectures tailored for robot fleet coordination, evaluating their design choices and scalability.

Findings

Robot-centric and graph-floor models perform best in prediction tasks.

Models benefit from larger datasets and scale effectively.

Localized interaction structures improve model performance.

Abstract

We introduce DeepFleet, a suite of foundation models designed to support coordination and planning for large-scale mobile robot fleets. These models are trained on fleet movement data, including robot positions, goals, and interactions, from hundreds of thousands of robots in Amazon warehouses worldwide. DeepFleet consists of four architectures that each embody a distinct inductive bias and collectively explore key points in the design space for multi-agent foundation models: the robot-centric (RC) model is an autoregressive decision transformer operating on neighborhoods of individual robots; the robot-floor (RF) model uses a transformer with cross-attention between robots and the warehouse floor; the image-floor (IF) model applies convolutional encoding to a multi-channel image representation of the full fleet; and the graph-floor (GF) model combines temporal attention with graph neural networks for spatial relationships. In this paper, we describe these models and present our evaluation of the impact of these design choices on prediction task performance. We find that the robot-centric and graph-floor models, which both use asynchronous robot state updates and incorporate the localized structure of robot interactions, show the most promise. We also present experiments that show that these two models can make effective use of larger warehouses operation datasets as the models are scaled up.