Generalizable Collaborative Search-and-Capture in Cluttered Environments via Path-Guided MAPPO and Directional Frontier Allocation

TL;DR

This paper introduces PGF-MAPPO, a hierarchical multi-agent reinforcement learning framework that enhances collaborative pursuit in cluttered environments through path-guided planning, reward shaping, and directional frontier allocation, achieving robust generalization and efficiency.

Contribution

The paper presents a novel hierarchical framework combining topological planning with reactive control, integrating A*-based reward shaping and directional frontier allocation for scalable, generalizable pursuit-evasion.

Findings

Achieves superior capture efficiency compared to baselines.

Demonstrates robust zero-shot generalization to larger environments.

Maintains low model complexity suitable for robotic swarms.

Abstract

Collaborative pursuit-evasion in cluttered environments presents significant challenges due to sparse rewards and constrained Fields of View (FOV). Standard Multi-Agent Reinforcement Learning (MARL) often suffers from inefficient exploration and fails to scale to large scenarios. We propose PGF-MAPPO (Path-Guided Frontier MAPPO), a hierarchical framework bridging topological planning with reactive control. To resolve local minima and sparse rewards, we integrate an A*-based potential field for dense reward shaping. Furthermore, we introduce Directional Frontier Allocation, combining Farthest Point Sampling (FPS) with geometric angle suppression to enforce spatial dispersion and accelerate coverage. The architecture employs a parameter-shared decentralized critic, maintaining O(1) model complexity suitable for robotic swarms. Experiments demonstrate that PGF-MAPPO achieves superior capture efficiency against faster evaders. Policies trained on 10x10 maps exhibit robust zero-shot generalization to unseen 20x20 environments, significantly outperforming rule-based and learning-based baselines.