Toward Scalable Multirobot Control: Fast Policy Learning in Distributed MPC

TL;DR

This paper introduces a fast, scalable distributed policy learning framework for multirobot control that generates explicit control policies without heavy online optimization, enabling real-time deployment in large-scale systems.

Contribution

The paper presents a novel distributed policy learning algorithm for multirobot control that improves scalability and efficiency over traditional DMPC methods by avoiding numerical solvers.

Findings

Effective control policy learning for systems with up to 10,000 robots.

Demonstrated scalability and real-time deployment in large multirobot systems.

Enhanced transferability of policies across different robot scales.

Abstract

Distributed model predictive control (DMPC) is promising in achieving optimal cooperative control in multirobot systems (MRS). However, real-time DMPC implementation relies on numerical optimization tools to periodically calculate local control sequences online. This process is computationally demanding and lacks scalability for large-scale, nonlinear MRS. This article proposes a novel distributed learning-based predictive control (DLPC) framework for scalable multirobot control. Unlike conventional DMPC methods that calculate open-loop control sequences, our approach centers around a computationally fast and efficient distributed policy learning algorithm that generates explicit closed-loop DMPC policies for MRS without using numerical solvers. The policy learning is executed incrementally and forward in time in each prediction interval through an online distributed actor-critic implementation. The control policies are successively updated in a receding-horizon manner, enabling fast and efficient policy learning with the closed-loop stability guarantee. The learned control policies could be deployed online to MRS with varying robot scales, enhancing scalability and transferability for large-scale MRS. Furthermore, we extend our methodology to address the multirobot safe learning challenge through a force field-inspired policy learning approach. We validate our approach's effectiveness, scalability, and efficiency through extensive experiments on cooperative tasks of large-scale wheeled robots and multirotor drones. Our results demonstrate the rapid learning and deployment of DMPC policies for MRS with scales up to 10,000 units.