Multi-AUV Cooperative Underwater Multi-Target Tracking Based on Dynamic-Switching-enabled Multi-Agent Reinforcement Learning

TL;DR

This paper introduces a novel multi-AUV cooperative tracking algorithm using dynamic-switching multi-agent reinforcement learning, accounting for underwater factors, and demonstrates its effectiveness in precise multi-target underwater tracking.

Contribution

It presents a hierarchical SDN-based reinforcement learning architecture with dynamic-switching mechanisms and reward reshaping for improved convergence and tracking accuracy.

Findings

Enhanced tracking precision in underwater environments.

Faster convergence of the reinforcement learning algorithm.

Outperforms recent methods in key metrics.

Abstract

In recent years, autonomous underwater vehicle (AUV) swarms are gradually becoming popular and have been widely promoted in ocean exploration or underwater tracking, etc. In this paper, we propose a multi-AUV cooperative underwater multi-target tracking algorithm especially when the real underwater factors are taken into account. We first give normally modelling approach for the underwater sonar-based detection and the ocean current interference on the target tracking process. Then, based on software-defined networking (SDN), we regard the AUV swarm as a underwater ad-hoc network and propose a hierarchical software-defined multi-AUV reinforcement learning (HSARL) architecture. Based on the proposed HSARL architecture, we propose the "Dynamic-Switching" mechanism, it includes "Dynamic-Switching Attention" and "Dynamic-Switching Resampling" mechanisms which accelerate the HSARL algorithm's convergence speed and effectively prevents it from getting stuck in a local optimum state. Additionally, we introduce the reward reshaping mechanism for further accelerating the convergence speed of the proposed HSARL algorithm in early phase. Finally, based on a proposed AUV classification method, we propose a cooperative tracking algorithm called Dynamic-Switching-Based MARL (DSBM)-driven tracking algorithm. Evaluation results demonstrate that our proposed DSBM tracking algorithm can perform precise underwater multi-target tracking, comparing with many of recent research products in terms of various important metrics.