Sparse shepherding control of large-scale multi-agent systems via Reinforcement Learning

TL;DR

This paper introduces a Reinforcement Learning framework for controlling large multi-agent systems sparsely by coupling microscopic and macroscopic models, achieving target distributions efficiently and robustly.

Contribution

It presents a novel RL-based sparse control method combining ODE-PDE coupling and adaptive interaction compensation for large-scale multi-agent systems.

Findings

Effective density control demonstrated in simulations

Achieves target distributions with robustness to disturbances

Reduces reliance on online optimization

Abstract

We propose a Reinforcement Learning framework for sparse indirect control of large-scale multi-agent systems, where few controlled agents shape the collective behavior of many uncontrolled agents. The approach addresses this multi-scale challenge by coupling ODEs (modeling controlled agents) with a PDE (describing the uncontrolled population density), capturing how microscopic control achieves macroscopic objectives. Our method combines model-free Reinforcement Learning with adaptive interaction strength compensation to overcome sparse actuation limitations. Numerical validation demonstrates effective density control, with the system achieving target distributions while maintaining robustness to disturbances and measurement noise, confirming that learning-based sparse control can replace computationally expensive online optimization.