# A Q-Learning-Based Distributed Energy-Efficient Routing Protocol in UASNs

## TL;DR

This paper introduces a new routing protocol for underwater sensor networks that uses machine learning to save energy and improve performance.

## Contribution

The novel contribution is a distributed Q-learning protocol that incorporates node energy and link quality for efficient routing in UASNs.

## Key findings

- The QDER protocol outperforms existing methods in network lifetime and energy efficiency.
- The protocol shows robustness and adaptability under varying signal-to-noise ratio conditions.
- Distributed decision-making reduces communication overhead compared to centralized approaches.

## Abstract

This paper proposes a Q-Learning-Based Distributed Energy-Efficient Routing (QDER) protocol for underwater acoustic sensor networks (UASNs). The routing problem is formulated as a Markov Decision Process (MDP) and a distributed Q-learning approach is proposed. Each sensor node is treated as an agent that independently selects its next-hop node based on a Q-table. The rewards function is designed that jointly considers node residual energy and depth information, enabling each node to learn an effective routing policy through distributed decision-making. Unlike centralized routing approaches that rely on extensive global information exchange, the proposed scheme allows nodes to make local decisions, thereby reducing communication overhead and energy consumption while maintaining efficient routing paths. In addition, link quality is designed in the reward to account for channel conditions, which improves the robustness of the routing strategy under noisy underwater acoustic environments. Simulation results demonstrate that the QDER achieves better system performance compared with Depth-Based Routing (DBR) and Deep Q-Network-Based Intelligent Routing (DQIR). Considering channel attenuation and noise, the proposed method with the link quality metric achieves improved network lifetime and energy efficiency. It also shows good robustness and adaptability under different signal-to-noise ratio (SNR) conditions.