Distributed Load Balancing with Workload-Dependent Service Rates

TL;DR

This paper introduces a distributed routing policy for bipartite queueing systems with workload-dependent service rates, achieving near-optimal load balancing and stability without requiring knowledge of arrival rates.

Contribution

The paper proposes the GMSR policy that converges to optimal routing behavior in stochastic and fluid models, ensuring stability and throughput maximization in distributed load balancing.

Findings

GMSR policy converges almost surely to a fluid model behavior.

GMSR attains $oldsymbol{ ext{ extit{epsilon}}}$-suboptimality in $O( ext{ extit{delta}} + ext{log}(1/ ext{ extit{epsilon}}))$ time.

GMSR maximizes throughput and stability in overloaded systems.

Abstract

We study distributed load balancing in bipartite queueing systems where frontends route jobs to heterogeneous backends with workload-dependent service rates. The system's connectivity -- governed by compatibility constraints such as data residency or resource requirements -- is represented by an arbitrary bipartite graph. Each frontend operates independently without communication with other frontends, and the goal is to minimize the expected average latency of all jobs. We propose a closed-loop policy called the Greatest Marginal Service Rate (GMSR) policy that achieves effective coordination without requiring knowledge of arrival rates. In a discrete-time stochastic model, we show that the behavior of our routing policy converges (almost surely) to the behavior of a fluid model, in the limit as job sizes tend to zero and job arrival rates are scaled so that the expected total volume of jobs arriving per unit time remains fixed. Then, in the fluid regime, we demonstrate that the policy attains an $\epsilon$-suboptimal solution in $O(\delta + \log{1/\epsilon})$ time from $\delta$-suboptimal initial workloads, which implies global convergence to the centrally coordinated optimal routing. Finally, we analyze the fluid model when the system is overloaded. We show that GMSR lexicographically maximizes throughput, maximizes the number of stable backends, and minimizes their collective workload.