Flexible Queueing Architectures

TL;DR

This paper investigates multi-server queueing systems with flexible server-queue connectivity, demonstrating that limited flexibility can still achieve near-optimal capacity and low delays using expander-graph-based architectures and novel scheduling policies.

Contribution

It introduces a new analysis of expander-graph-based flexible architectures achieving near-optimal capacity and delay performance with limited connectivity, along with a novel virtual-queue scheduling policy.

Findings

Expander-graph architectures achieve capacity within a constant factor of the maximum.

Limited flexibility ($d_n o ext{large but } d_n ot o n$) suffices for low delays.

A new virtual-queue scheduling policy effectively manages job-server assignments.

Abstract

We study a multi-server model with $n$ flexible servers and $n$ queues, connected through a bipartite graph, where the level of flexibility is captured by the graph's average degree, $d_n$. Applications in content replication in data centers, skill-based routing in call centers, and flexible supply chains are among our main motivations. We focus on the scaling regime where the system size $n$ tends to infinity, while the overall traffic intensity stays fixed. We show that a large capacity region and an asymptotically vanishing queueing delay are simultaneously achievable even under limited flexibility ($d_n \ll n$). Our main results demonstrate that, when $d_n\gg \ln n$, a family of expander-graph-based flexibility architectures has a capacity region that is within a constant factor of the maximum possible, while simultaneously ensuring a diminishing queueing delay for all arrival rate vectors in the capacity region. Our analysis is centered around a new class of virtual-queue-based scheduling policies that rely on dynamically constructed job-to-server assignments on the connectivity graph. For comparison, we also analyze a natural family of modular architectures, which is simpler but has provably weaker performance.