Anypath Routing Protocol Design via Q-Learning for Underwater Sensor Networks

TL;DR

This paper introduces a Q-learning-based routing protocol for underwater sensor networks that aims to reduce latency and extend network lifetime by considering residual energy and depth information.

Contribution

It presents a novel localization-free anypath routing protocol using Q-learning with reward functions for energy and depth, improving delay and lifetime.

Findings

Reduces end-to-end delay in underwater sensor networks.

Extends network lifetime compared to existing protocols.

Demonstrates superior performance through extensive simulations.

Abstract

As a promising technology in the Internet of Underwater Things, underwater sensor networks have drawn a widespread attention from both academia and industry. However, designing a routing protocol for underwater sensor networks is a great challenge due to high energy consumption and large latency in the underwater environment. This paper proposes a Q-learning-based localization-free anypath routing (QLFR) protocol to prolong the lifetime as well as reduce the end-to-end delay for underwater sensor networks. Aiming at optimal routing policies, the Q-value is calculated by jointly considering the residual energy and depth information of sensor nodes throughout the routing process. More specifically, we define two reward functions (i.e., depth-related and energy-related rewards) for Q-learning with the objective of reducing latency and extending network lifetime. In addition, a new holding time mechanism for packet forwarding is designed according to the priority of forwarding candidate nodes. Furthermore, a mathematical analysis is presented to analyze the performance of the proposed routing protocol. Extensive simulation results demonstrate the superiority performance of the proposed routing protocol in terms of the end-to-end delay and the network lifetime.