Optimal Hyper-Scalable Load Balancing with a Strict Queue Limit

TL;DR

This paper investigates load balancing in large-scale systems under extremely limited communication and strict queue limits, proposing a universal throughput bound and a scheme that is proven to be throughput-optimal in such hyper-scalable regimes.

Contribution

It introduces a universal throughput upper bound for dispatcher algorithms under sparse communication and queue constraints, and presents a scheme that achieves this bound, proving throughput optimality.

Findings

Universal throughput bound established for sparse communication regimes.

Proposed scheme operates at any message rate and enforces queue limits.

Scheme proven to be throughput-optimal in many-server regimes.

Abstract

Load balancing plays a critical role in efficiently dispatching jobs in parallel-server systems such as cloud networks and data centers. A fundamental challenge in the design of load balancing algorithms is to achieve an optimal trade-off between delay performance and implementation overhead (e.g. communication or memory usage). This trade-off has primarily been studied so far from the angle of the amount of overhead required to achieve asymptotically optimal performance, particularly vanishing delay in large-scale systems. In contrast, in the present paper, we focus on an arbitrarily sparse communication budget, possibly well below the minimum requirement for vanishing delay, referred to as the hyper-scalable operating region. Furthermore, jobs may only be admitted when a specific limit on the queue position of the job can be guaranteed. The centerpiece of our analysis is a universal upper bound for the achievable throughput of any dispatcher-driven algorithm for a given communication budget and queue limit. We also propose a specific hyper-scalable scheme which can operate at any given message rate and enforce any given queue limit, while allowing the server states to be captured via a closed product-form network, in which servers act as customers traversing various nodes. The product-form distribution is leveraged to prove that the bound is tight and that the proposed hyper-scalable scheme is throughput-optimal in a many-server regime given the communication and queue limit constraints. Extensive simulation experiments are conducted to illustrate the results.