Delay-Fluctuation-Resistant Underwater Acoustic Network Access Method Based on Deep Reinforcement Learning

TL;DR

This paper introduces a new deep reinforcement learning method for underwater acoustic networks that handles delay fluctuations better than existing approaches.

Contribution

The novel approach integrates delay fluctuations into the state model and uses an optimized DDQN to improve learning efficiency and network performance.

Findings

The proposed method improves convergence speed by 29.3% and 15.5% compared to other DRL methods.

It significantly enhances normalized throughput compared to TDMA and DOTS protocols.

Abstract

The slow propagation speed of acoustic waves in water leads to significant variations and random fluctuations in communication delays among underwater acoustic sensor network (UASN) nodes. Conventional deep reinforcement learning (DRL)-based underwater acoustic network access methods can adaptively adjust their parameters and improve network communication efficiency by effectively utilizing inter-node delay differences for concurrent communication. However, they still suffer from shortcomings such as not accounting for random delay fluctuations in underwater acoustic links and low learning efficiency. This paper proposes a DRL-based delay-fluctuation-resistant underwater acoustic network access method. First, delay fluctuations are integrated into the state model of deep reinforcement learning, enabling the model to adapt to delay fluctuations during learning. Then, a double deep Q-network (DDQN) is introduced, and its structure is optimized to enhance learning and decision-making in complex environments. Simulations demonstrate that the proposed method achieves an average improvement of 29.3% and 15.5% in convergence speed compared to the other two DRL-based methods under varying delay fluctuations. Furthermore, the proposed method significantly enhances the normalized throughput compared to conventional Time Division Multiple Access (TDMA) and DOTS protocols.

Figures