ARBoids: Adaptive Residual Reinforcement Learning With Boids Model for Cooperative Multi-USV Target Defense

TL;DR

ARBoids introduces an adaptive reinforcement learning framework combined with a biologically inspired model to enhance multi-USV target defense, demonstrating improved interception performance and robustness in complex, high-maneuverability scenarios.

Contribution

This work presents a novel integration of deep reinforcement learning with the Boids model for adaptive multi-agent USV defense, addressing maneuverability challenges.

Findings

Outperforms traditional interception strategies in simulation

Exhibits strong adaptability to diverse attacker maneuvers

Demonstrates robustness and generalization in high-fidelity environments

Abstract

The target defense problem (TDP) for unmanned surface vehicles (USVs) concerns intercepting an adversarial USV before it breaches a designated target region, using one or more defending USVs. A particularly challenging scenario arises when the attacker exhibits superior maneuverability compared to the defenders, significantly complicating effective interception. To tackle this challenge, this letter introduces ARBoids, a novel adaptive residual reinforcement learning framework that integrates deep reinforcement learning (DRL) with the biologically inspired, force-based Boids model. Within this framework, the Boids model serves as a computationally efficient baseline policy for multi-agent coordination, while DRL learns a residual policy to adaptively refine and optimize the defenders' actions. The proposed approach is validated in a high-fidelity Gazebo simulation environment, demonstrating superior performance over traditional interception strategies, including pure force-based approaches and vanilla DRL policies. Furthermore, the learned policy exhibits strong adaptability to attackers with diverse maneuverability profiles, highlighting its robustness and generalization capability. The code of ARBoids will be released upon acceptance of this letter.