Power-of-$d$ Choices Load Balancing in the Sub-Halfin Whitt Regime

TL;DR

This paper analyzes the power-of-d choices load balancing algorithm in a large-scale, heavy-traffic setting, revealing how the queue length behavior depends on the growth rate of d relative to the number of servers.

Contribution

It characterizes the queue length distribution for various growth rates of d in the sub-Halfin-Whitt regime, extending previous results to smaller d values.

Findings

Queue length is approximately m when d is polynomial in n with exponent 1/m.

Queue length becomes unbounded when d grows polylogarithmically in n.

High probability bounds on queue lengths are established using iterative state space collapse.

Abstract

We consider the load balancing system under Poisson arrivals, exponential services, and homogeneous servers. Upon arrival, a job is to be routed to one of the servers, where it is queued until service. We consider the Power-of-$d$ choices routing algorithm, which chooses the queue with minimum length among $d$ randomly sampled queues. We study this system in the many-server heavy-traffic regime where the number of servers goes to infinity simultaneously when the load approaches the capacity. In particular, we consider a sequence of systems with $n$ servers and the arrival rate is given by $\lambda=n-n^{1-\gamma}$ for some $\gamma \in (0, 0.5)$, known as the sub-Halfin-Whitt regime. It was shown by [Liu Ying (2020)] that under Power-of-$d$ choices routing with $d \geq n^\gamma \log n$, the queue length behaves similarly to that of JSQ and that there are asymptotically zero queueing delays. The focus of this paper is to characterize the behavior when $d$ is below this threshold. We obtain high probability bounds on the queue lengths for various values of $d$ and large enough $n$. In particular, we show that when $d$ grows polynomially in $n$ but slower than in [Liu Ying (2020)], i.e., if $d$ is $\Theta\left((n^\gamma\log n)^{1/m})\right)$ for some integer $m>1$, then the asymptotic queue length is $m$ with high probability. Moreover, if $d$ grows polylog in $n$, i.e., slower than any polynomial, but is at least $\Omega(\log (n)^3)$, the queue length blows up to infinity asymptotically. We obtain these results by using an iterative state space collapse approach. We first establish a weak state-space collapse (SSC) on the queue lengths. Then, we bootstrap on weak SSC to iteratively narrow down the region of the collapse. After enough steps, this inductive refinement provides the bounds we seek. We establish these sequences of collapse using Lyapunov drift arguments.