M$^{2}$GRPO: Mamba-based Multi-Agent Group Relative Policy Optimization for Biomimetic Underwater Robots Pursuit

TL;DR

The paper introduces M$^{2}$GRPO, a novel multi-agent policy optimization framework for biomimetic underwater robots that enhances long-horizon decision making, inter-robot coordination, and stability, outperforming existing methods.

Contribution

It proposes a new Mamba-based group-relative policy optimization method that integrates attention mechanisms and reward normalization for scalable, stable multi-agent pursuit in underwater robots.

Findings

M$^{2}$GRPO outperforms MAPPO and recurrent baselines in pursuit success rate.

The method improves capture efficiency in simulated and real-world experiments.

It reduces training resource demands while maintaining stability and scalability.

Abstract

Traditional policy learning methods in cooperative pursuit face fundamental challenges in biomimetic underwater robots, where long-horizon decision making, partial observability, and inter-robot coordination require both expressiveness and stability. To address these issues, a novel framework called Mamba-based multi-agent group relative policy optimization (M$^{2}$GRPO) is proposed, which integrates a selective state-space Mamba policy with group-relative policy optimization under the centralized-training and decentralized-execution (CTDE) paradigm. Specifically, the Mamba-based policy leverages observation history to capture long-horizon temporal dependencies and exploits attention-based relational features to encode inter-agent interactions, producing bounded continuous actions through normalized Gaussian sampling. To further improve credit assignment without sacrificing stability, the group-relative advantages are obtained by normalizing rewards across agents within each episode and optimized through a multi-agent extension of GRPO, significantly reducing the demand for training resources while enabling stable and scalable policy updates. Extensive simulations and real-world pool experiments across team scales and evader strategies demonstrate that M$^{2}$GRPO consistently outperforms MAPPO and recurrent baselines in both pursuit success rate and capture efficiency. Overall, the proposed framework provides a practical and scalable solution for cooperative underwater pursuit with biomimetic robot systems.