Harvest-or-Transmit Policy for Cognitive Radio Networks: A Learning Theoretic Approach

TL;DR

This paper develops and compares learning-based, statistical, and offline optimization policies for energy-harvesting secondary users in cognitive radio networks to maximize throughput under different knowledge assumptions.

Contribution

It introduces a harvest-or-transmit policy with optimal power control using Q-learning, stochastic dynamic programming, and Benders decomposition, addressing different knowledge scenarios.

Findings

Q-learning-based online policy performs well without prior knowledge.

Optimal policies significantly improve throughput compared to baseline.

System parameters critically influence policy effectiveness.

Abstract

We consider an underlay cognitive radio network where the secondary user (SU) harvests energy from the environment. We consider a slotted-mode of operation where each slot of SU is used for either energy harvesting or data transmission. Considering block fading with memory, we model the energy arrival and fading processes as a stationary Markov process of first order. We propose a harvest-or-transmit policy for the SU along with optimal transmit powers that maximize its expected throughput under three different settings. First, we consider a learning-theoretic approach where we do not assume any apriori knowledge about the underlying Markov processes. In this case, we obtain an online policy using Q-learning. Then, we assume that the full statistical knowledge of the governing Markov process is known apriori. Under this assumption, we obtain an optimal online policy using infinite horizon stochastic dynamic programming. Finally, we obtain an optimal offline policy using the generalized Benders decomposition algorithm. The offline policy assumes that for a given time deadline, the energy arrivals and channel states are known in advance at all the transmitters. Finally, we compare all policies and study the effects of various system parameters on the system performance.