Decentralized Multi-Agent Pursuit using Deep Reinforcement Learning

TL;DR

This paper presents a deep reinforcement learning approach for multi-agent pursuit with non-holonomic constraints, achieving competitive performance in simulation and real-world drone experiments.

Contribution

It introduces a novel training framework for pursuit with unicycle agents, combining curriculum learning and group rewards, and demonstrates successful transfer to real drones.

Findings

RL-based pursuit matches classical algorithms with omni-directional agents

Outperforms adapted classical algorithms for non-holonomic agents

Successfully transfers learned policy from simulation to real drones

Abstract

Pursuit-evasion is the problem of capturing mobile targets with one or more pursuers. We use deep reinforcement learning for pursuing an omni-directional target with multiple, homogeneous agents that are subject to unicycle kinematic constraints. We use shared experience to train a policy for a given number of pursuers that is executed independently by each agent at run-time. The training benefits from curriculum learning, a sweeping-angle ordering to locally represent neighboring agents and encouraging good formations with reward structure that combines individual and group rewards. Simulated experiments with a reactive evader and up to eight pursuers show that our learning-based approach, with non-holonomic agents, performs on par with classical algorithms with omni-directional agents, and outperforms their non-holonomic adaptations. The learned policy is successfully transferred to the real world in a proof-of-concept demonstration with three motion-constrained pursuer drones.