Optimal Rate-Matrix Pruning For Large-Scale Heterogeneous Systems

TL;DR

This paper introduces two asymptotically delay-optimal load balancing policies for large-scale heterogeneous systems, addressing the limitations of existing policies and achieving near-zero queuing in the asymptotic regime.

Contribution

The paper proposes and analyzes two novel load balancing policies that are asymptotically delay-optimal in large heterogeneous systems, filling a key gap in scalable, provably efficient strategies.

Findings

Both policies achieve asymptotic zero queuing.

The probability of assigning tasks to idle servers approaches 1 as system scales.

The policies outperform traditional load balancing methods in heterogeneous environments.

Abstract

We present an analysis of large-scale load balancing systems, where the processing time distribution of tasks depends on both the task and server types. Our study focuses on the asymptotic regime, where the number of servers and task types tend to infinity in proportion. In heterogeneous environments, commonly used load balancing policies such as Join Fastest Idle Queue and Join Fastest Shortest Queue exhibit poor performance and even shrink the stability region. Interestingly, prior to this work, finding a scalable policy with a provable performance guarantee in this setup remained an open question. To address this gap, we propose and analyze two asymptotically delay-optimal dynamic load balancing policies. The first policy efficiently reserves the processing capacity of each server for ``good" tasks and routes tasks using the vanilla Join Idle Queue policy. The second policy, called the speed-priority policy, significantly increases the likelihood of assigning tasks to the respective ``good" servers capable of processing them at high speeds. By leveraging a framework inspired by the graphon literature and employing the mean-field method and stochastic coupling arguments, we demonstrate that both policies achieve asymptotic zero queuing. Specifically, as the system scales, the probability of a typical task being assigned to an idle server approaches 1.