Universality of Load Balancing Schemes on Diffusion Scale

TL;DR

This paper analyzes load balancing schemes in large-scale queue systems, showing that in heavy traffic, the specific choice of sampling parameter does not affect the diffusion limit, highlighting the sufficiency of assigning tasks to idle servers.

Contribution

The paper proves that the diffusion limit of load balancing schemes in the Halfin-Whitt regime is independent of the sampling parameter, unifying different policies under a common diffusion behavior.

Findings

Diffusion limit is independent of the sampling parameter d.

Assigning tasks to idle servers achieves diffusion-level optimality.

The analysis applies to a broad class of load balancing policies.

Abstract

We consider a system of $N$ parallel queues with identical exponential service rates and a single dispatcher where tasks arrive as a Poisson process. When a task arrives, the dispatcher always assigns it to an idle server, if there is any, and to a server with the shortest queue among $d$ randomly selected servers otherwise $(1 \leq d \leq N)$. This load balancing scheme subsumes the so-called Join-the-Idle Queue (JIQ) policy $(d = 1)$ and the celebrated Join-the-Shortest Queue (JSQ) policy $(d = N)$ as two crucial special cases. We develop a stochastic coupling construction to obtain the diffusion limit of the queue process in the Halfin-Whitt heavy-traffic regime, and establish that it does not depend on the value of $d$, implying that assigning tasks to idle servers is sufficient for diffusion level optimality.