Joint AP Probing and Scheduling: A Contextual Bandit Approach

TL;DR

This paper introduces a novel contextual bandit model with probing capabilities for joint access point scheduling, enabling better decision-making under uncertain data rates in wireless networks.

Contribution

It proposes a new CBwP model that extends classic bandit frameworks to include probing, with an efficient algorithm and regret analysis for Bernoulli data rates.

Findings

Developed an efficient algorithm for CBwP.

Established regret bounds for Bernoulli data rates.

Demonstrated potential applications in sequential decision-making.

Abstract

We consider a set of APs with unknown data rates that cooperatively serve a mobile client. The data rate of each link is i.i.d. sampled from a distribution that is unknown a priori. In contrast to traditional link scheduling problems under uncertainty, we assume that in each time step, the device can probe a subset of links before deciding which one to use. We model this problem as a contextual bandit problem with probing (CBwP) and present an efficient algorithm. We further establish the regret of our algorithm for links with Bernoulli data rates. Our CBwP model is a novel extension of the classic contextual bandit model and can potentially be applied to a large class of sequential decision-making problems that involve joint probing and play under uncertainty.