A lower bound on the queueing delay in resource constrained load balancing

TL;DR

This paper establishes a fundamental lower bound on the resources needed for load balancing policies to achieve vanishing queueing delay in large-scale distributed systems, showing that certain resource constraints prevent delay reduction.

Contribution

It proves that symmetric dispatching policies with limited memory and message exchange cannot achieve zero queueing delay as the system scales.

Findings

Finite memory and message rate policies cannot eliminate queueing delay asymptotically.

Expected queueing delay remains bounded away from zero under certain resource constraints.

Symmetric policies with minimal resources are insufficient for delay minimization in large systems.

Abstract

We consider the following distributed service model: jobs with unit mean, general distribution, and independent processing times arrive as a renewal process of rate $\lambda n$, with $0<\lambda<1$, and are immediately dispatched to one of several queues associated with $n$ identical servers with unit processing rate. We assume that the dispatching decisions are made by a central dispatcher endowed with a finite memory, and with the ability to exchange messages with the servers. We study the fundamental resource requirements (memory bits and message exchange rate), in order to drive the expected queueing delay in steady-state of a typical job to zero, as $n$ increases. We develop a novel approach to show that, within a certain broad class of "symmetric" policies, every dispatching policy with a message rate of the order of $n$, and with a memory of the order of $\log n$ bits, results in an expected queueing delay which is bounded away from zero, uniformly as $n\to\infty$.