Whittle Index Based User Association in Dense Millimeter Wave Networks

TL;DR

This paper develops a Whittle index-based user association scheme for dense millimeter wave networks, transforming a complex problem into a decoupled Markov Decision Process and demonstrating superior performance through extensive simulations.

Contribution

It introduces a novel Whittle index-based approach for user association in mmWave networks, proving indexability and providing a practical scheme that outperforms existing methods.

Findings

Outperforms prior schemes in average cost, delay, throughput, and fairness.

Proves the problem is Whittle indexable and provides a computation scheme.

Demonstrates effectiveness through extensive simulations.

Abstract

We address the problem of user association in a dense millimeter wave (mmWave) network, in which each arriving user brings a file containing a random number of packets and each time slot is divided into multiple mini-slots. This problem is an instance of the restless multi-armed bandit problem, and is provably hard to solve. Using a technique introduced by Whittle, we relax the hard per-stage constraint that each arriving user must be associated with exactly one mmWave base station (mBS) to a long-term constraint and then use the Lagrangian multiplier technique to convert the problem into an unconstrained problem. This decouples the process governing the system into separate Markov Decision Processes at different mBSs. We prove that the problem is Whittle indexable, present a scheme for computing the Whittle indices of different mBSs, and propose an association scheme under which, each arriving user is associated with the mBS with the smallest value of the Whittle index. Using extensive simulations, we show that the proposed Whittle index based scheme outperforms several user association schemes proposed in prior work in terms of various performance metrics such as average cost, delay, throughput, and Jain's fairness index.