Underwater Differential Game: Finite-Time Target Hunting Task with Communication Delay

TL;DR

This paper develops a differential game framework combined with deep reinforcement learning to optimize underwater target hunting by UUV swarms, accounting for communication delays and disturbances in finite time.

Contribution

It introduces a novel approach integrating differential game theory with DRL to handle underwater communication delays and disturbances in target hunting tasks.

Findings

Disturbances significantly affect system performance.

UUVs maintain better consistency with smaller disturbances.

Simulation results validate the effectiveness of the proposed method.

Abstract

This work considers designing an unmanned target hunting system for a swarm of unmanned underwater vehicles (UUVs) to hunt a target with high maneuverability. Differential game theory is used to analyze combat policies of UUVs and the target within finite time. The challenge lies in UUVs must conduct their control policies in consideration of not only the consistency of the hunting team but also escaping behaviors of the target. To obtain stable feedback control policies satisfying Nash equilibrium, we construct the Hamiltonian function with Leibniz's formula. For further taken underwater disturbances and communication delay into consideration, modified deep reinforcement learning (DRL) is provided to investigate the underwater target hunting task in an unknown dynamic environment. Simulations show that underwater disturbances have a large impact on the system considering communication delay. Moreover, consistency tests show that UUVs perform better consistency with a relatively small range of disturbances.