Reinforcement Learning for Secrecy Optimization in Underwater Energy Harvesting Relay Network

TL;DR

This paper proposes a reinforcement learning-based power allocation strategy for underwater energy-harvesting relay networks supporting hybrid optical-acoustic communication, enhancing long-term secrecy performance amid dynamic underwater conditions.

Contribution

It introduces a model-based RL approach for optimal power allocation in underwater EH relay networks with hybrid links, addressing secrecy and channel variability.

Findings

RL-based OPA outperforms greedy and naive algorithms in secrecy performance

RL adapts effectively to battery and channel dynamics

Naive algorithm performs poorly due to short-sighted decisions

Abstract

This paper explores secure communication in an underwater energy-harvesting (EH) relay network that supports hybrid optical-acoustic transmission. The optical hop is modeled using a Gamma-Gamma turbulence channel with pointing errors and may occasionally be blocked by underwater obstacles. At the same time, an eavesdropper is assumed to monitor the acoustic hop, creating a secrecy concern. To address this, we formulate the relay power allocation problem as an infinite-horizon Markov decision process (MDP). A model-based reinforcement learning (RL) driven optimal power allocation (OPA) strategy is proposed to maximize long-term cumulative secrecy performance until the network stops functioning. To offer lower-complexity alternatives, we also develop a Greedy Algorithm (GA) and a Naive Algorithm (NA). Simulation results show that the RL based OPA adapts effectively to battery dynamics, varying channel conditions, and optical link availability, achieving the highest secure data transmission, while GA performs reasonably and NA performs poorly due to its short-sighted decisions.