On The Optimality of Myopic Sensing in Multi-State Channels

TL;DR

This paper investigates the optimality of myopic sensing policies in multi-state Markov channel models, providing conditions under which such simple policies are optimal, and linking them to the Gittins index rule.

Contribution

The paper establishes sufficient conditions for the optimality of myopic sensing policies in multi-state channels and connects these policies to the Gittins index rule.

Findings

Identifies conditions guaranteeing myopic policy optimality.

Shows myopic policy coincides with Gittins index under certain conditions.

Provides insights into restless bandit problem solutions.

Abstract

We consider the channel sensing problem arising in opportunistic scheduling over fading channels, cognitive radio networks, and resource constrained jamming. The communication system consists of N channels. Each channel is modeled as a multi-state Markov chain (M.C.). At each time instant a user selects one channel to sense and uses it to transmit information. A reward depending on the state of the selected channel is obtained for each transmission. The objective is to design a channel sensing policy that maximizes the expected total reward collected over a finite or infinite horizon. This problem can be viewed as an instance of a restless bandit problem, for which the form of optimal policies is unknown in general. We discover sets of conditions sufficient to guarantee the optimality of a myopic sensing policy; we show that under one particular set of conditions the myopic policy coincides with the Gittins index rule.