Load Balancing in the Non-Degenerate Slowdown Regime

TL;DR

This paper analyzes the Join-the-Shortest-Queue policy in the Non-Degenerate Slowdown regime, providing new diffusion approximations and comparing its performance with other load balancing policies, leading to the design of more efficient rules.

Contribution

It introduces a novel diffusion approximation and timescale separation for JSQ in the NDS regime, and proposes new load balancing rules with similar performance but lower communication costs.

Findings

Diffusion approximation for JSQ in NDS regime

Quantification of the cost of irrevocable dispatching

Development of low-communication load balancing rules

Abstract

We analyse Join-the-Shortest-Queue in a contemporary scaling regime known as the Non-Degenerate Slowdown regime. Join-the-Shortest-Queue (JSQ) is a classical load balancing policy for queueing systems with multiple parallel servers. Parallel server queueing systems are regularly analysed and dimensioned by diffusion approximations achieved in the Halfin-Whitt scaling regime. However, when jobs must be dispatched to a server upon arrival, we advocate the Non-Degenerate Slowdown regime (NDS) to compare different load-balancing rules. In this paper we identify novel diffusion approximation and timescale separation that provides insights into the performance of JSQ. We calculate the price of irrevocably dispatching jobs to servers and prove this to within 15% (in the NDS regime) of the rules that may manoeuvre jobs between servers. We also compare ours results for the JSQ policy with the NDS approximations of many modern load balancing policies such as Idle-Queue-First and Power-of-$d$-choices policies which act as low information proxies for the JSQ policy. Our analysis leads us to construct new rules that have identical performance to JSQ but require less communication overhead than power-of-2-choices.