Regret Bounds for Opportunistic Channel Access

TL;DR

This paper introduces an algorithm for opportunistic channel access in systems with unknown statistics, balancing exploration and exploitation, and provides regret bounds and performance evaluations for the approach.

Contribution

It formulates the problem as a POMDP and develops a novel algorithm with finite horizon regret bounds for unknown channel statistics.

Findings

Algorithm achieves near-optimal tradeoff between exploration and exploitation.

Finite horizon regret bounds are established for the proposed method.

Numerical results demonstrate effective performance in single and multiple channel scenarios.

Abstract

We consider the task of opportunistic channel access in a primary system composed of independent Gilbert-Elliot channels where the secondary (or opportunistic) user does not dispose of a priori information regarding the statistical characteristics of the system. It is shown that this problem may be cast into the framework of model-based learning in a specific class of Partially Observed Markov Decision Processes (POMDPs) for which we introduce an algorithm aimed at striking an optimal tradeoff between the exploration (or estimation) and exploitation requirements. We provide finite horizon regret bounds for this algorithm as well as a numerical evaluation of its performance in the single channel model as well as in the case of stochastically identical channels.