An order optimal policy for exploiting idle spectrum in cognitive radio networks

TL;DR

This paper introduces an order-optimal spectrum sensing policy for cognitive radio networks that balances exploration and exploitation, achieving logarithmic regret in dynamic spectrum access scenarios.

Contribution

The paper proposes a recency-based exploration policy for multi-band spectrum sensing modeled as a restless bandit problem, achieving asymptotically optimal regret.

Findings

Policy attains asymptotically logarithmic weak regret

Simulation results confirm the theoretical regret bounds

Policy outperforms existing methods at low complexity

Abstract

In this paper a spectrum sensing policy employing recency-based exploration is proposed for cognitive radio networks. We formulate the problem of finding a spectrum sensing policy for multi-band dynamic spectrum access as a stochastic restless multi-armed bandit problem with stationary unknown reward distributions. In cognitive radio networks the multi-armed bandit problem arises when deciding where in the radio spectrum to look for idle frequencies that could be efficiently exploited for data transmission. We consider two models for the dynamics of the frequency bands: 1) the independent model where the state of the band evolves randomly independently from the past and 2) the Gilbert-Elliot model, where the states evolve according to a 2-state Markov chain. It is shown that in these conditions the proposed sensing policy attains asymptotically logarithmic weak regret. The policy proposed in this paper is an index policy, in which the index of a frequency band is comprised of a sample mean term and a recency-based exploration bonus term. The sample mean promotes spectrum exploitation whereas the exploration bonus encourages for further exploration for idle bands providing high data rates. The proposed recency based approach readily allows constructing the exploration bonus such that it will grow the time interval between consecutive sensing time instants of a suboptimal band exponentially, which then leads to logarithmically increasing weak regret. Simulation results confirming logarithmic weak regret are presented and it is found that the proposed policy provides often improved performance at low complexity over other state-of-the-art policies in the literature.