Learning Efficient Flocking Control based on Gibbs Random Fields

TL;DR

This paper introduces a novel MARL framework based on Gibbs Random Fields for efficient, safe, and scalable flocking control in multi-robot systems, achieving high success rates in complex environments.

Contribution

It presents a new GRF-based MARL approach with decentralized training, an action attention module, and improved scalability for multi-robot flocking control.

Findings

Success rate around 99% in simulations and experiments

Enhanced scalability with decentralized training and GRF-based credit assignment

Validated efficiency through ablation studies

Abstract

Flocking control is essential for multi-robot systems in diverse applications, yet achieving efficient flocking in congested environments poses challenges regarding computation burdens, performance optimality, and motion safety. This paper addresses these challenges through a multi-agent reinforcement learning (MARL) framework built on Gibbs Random Fields (GRFs). With GRFs, a multi-robot system is represented by a set of random variables conforming to a joint probability distribution, thus offering a fresh perspective on flocking reward design. A decentralized training and execution mechanism, which enhances the scalability of MARL concerning robot quantity, is realized using a GRF-based credit assignment method. An action attention module is introduced to implicitly anticipate the motion intentions of neighboring robots, consequently mitigating potential non-stationarity issues in MARL. The proposed framework enables learning an efficient distributed control policy for multi-robot systems in challenging environments with success rate around $99\%$, as demonstrated through thorough comparisons with state-of-the-art solutions in simulations and experiments. Ablation studies are also performed to validate the efficiency of different framework modules.