Decentralized End-to-End Multi-AAV Pursuit Using Predictive Spatio-Temporal Observation via Deep Reinforcement Learning

TL;DR

This paper introduces a decentralized reinforcement learning framework for multi-AAV pursuit that uses raw LiDAR data and a novel predictive spatio-temporal observation to improve navigation and interception in cluttered environments.

Contribution

It presents a new end-to-end MARL approach with PSTO representation, enabling scalable, robust multi-agent pursuit without relying on privileged information.

Findings

Achieves higher capture efficiency than existing methods.

Scales seamlessly across different team sizes.

Validated in outdoor quadrotor experiments using only onboard sensors.

Abstract

Decentralized cooperative pursuit in cluttered environments is challenging for autonomous aerial swarms, especially under partial and noisy perception. Existing methods often rely on abstracted geometric features or privileged ground-truth states, and therefore sidestep perceptual uncertainty in real-world settings. We propose a decentralized end-to-end multi-agent reinforcement learning (MARL) framework that maps raw LiDAR observations directly to continuous control commands. Central to the framework is the Predictive Spatio-Temporal Observation (PSTO), an egocentric grid representation that aligns obstacle geometry with predictive adversarial intent and teammate motion in a unified, fixed-resolution projection. Built on PSTO, a single decentralized policy enables agents to navigate static obstacles, intercept dynamic targets, and maintain cooperative encirclement. Simulations demonstrate that the proposed method achieves superior capture efficiency and competitive success rates compared to state-of-the-art learning-based approaches relying on privileged obstacle information. Furthermore, the unified policy scales seamlessly across different team sizes without retraining. Finally, fully autonomous outdoor experiments validate the framework on a quadrotor swarm relying on only onboard sensing and computing.