Reinforcement Learning Based Robust Policy Design for Relay and Power Optimization in DF Relaying Networks

TL;DR

This paper introduces a reinforcement learning approach for relay and power optimization in DF relaying networks, focusing on robustness against environmental uncertainties and outperforming traditional RL methods in unseen scenarios.

Contribution

It proposes a robust RL algorithm for outage minimization that does not require prior channel knowledge and enhances worst-case performance in uncertain environments.

Findings

Improved worst-case outage performance by about 6%

Better generalization in unseen environments

Provides a lower bound on worst-case performance of RL policies

Abstract

In this paper, we study the outage minimization problem in a decode-and-forward cooperative network with relay uncertainty. To reduce the outage probability and improve the quality of service, existing researches usually rely on the assumption of both exact instantaneous channel state information (CSI) and environmental uncertainty. However, it is difficult to obtain perfect instantaneous CSI immediately under practical situations where channel states change rapidly, and the uncertainty in communication environments may not be observed, which makes traditional methods not applicable. Therefore, we turn to reinforcement learning (RL) methods for solutions, which do not need any prior knowledge of underlying channel or assumptions of environmental uncertainty. RL method is to learn from the interaction with communication environment, optimize its action policy, and then propose relay selection and power allocation schemes. We first analyse the robustness of RL action policy by giving the lower bound of the worst-case performance, when RL methods are applied to communication scenarios with environment uncertainty. Then, we propose a robust algorithm for outage probability minimization based on RL. Simulation results reveal that compared with traditional RL methods, our approach has better generalization ability and can improve the worst-case performance by about 6% when evaluated in unseen environments.