Optimal WiFi Sensing via Dynamic Programming

TL;DR

This paper develops a dynamic programming approach to optimize WiFi sensing schedules for mobile devices, addressing both known and unknown ON/OFF period distributions, with proven convergence and learning algorithms.

Contribution

It introduces a dynamic programming framework for optimal WiFi sensing, including solutions for exponential OFF periods and a bandit-based learning algorithm for unknown distributions.

Findings

Optimal sensing policies are explicitly characterized for exponential OFF periods.

The proposed learning algorithm achieves vanishing regret in unknown distribution scenarios.

Convergence of value iterations to the optimal solution is demonstrated.

Abstract

The problem of finding an optimal sensing schedule for a mobile device that encounters an intermittent WiFi access opportunity is considered. At any given time, the WiFi is in any of the two modes, ON or OFF, and the mobile's incentive is to connect to the WiFi in the ON mode as soon as possible, while spending as little sensing energy. We introduce a dynamic programming framework which enables the characterization of an explicit solution for several models, particularly when the OFF periods are exponentially distributed. While the problem for non-exponential OFF periods is ill-posed in general, a usual workaround in literature is to make the mobile device aware if one ON period is completely missed. In this restricted setting, using the DP framework, the deterministic nature of the optimal sensing policy is established, and value iterations are shown to converge to the optimal solution. Finally, we address the blind situation where the distributions of ON and OFF periods are unknown. A continuous bandit based learning algorithm that has vanishing regret (loss compared to the optimal strategy with the knowledge of distributions) is presented, and comparisons with the optimal schemes are provided for exponential ON and OFF times.