Decentralized Learning in Online Queuing Systems

TL;DR

This paper investigates decentralized learning strategies in online queuing systems, demonstrating that cooperation enables system stability at lower service-to-arrival rate ratios, approaching centralized performance.

Contribution

It introduces the first decentralized learning algorithm that guarantees stability for ratios above 1, improving upon previous no-regret strategies that required higher ratios.

Findings

Decentralized learning can achieve stability with ratio > 1.

Cooperation among queues is essential for stability at lower ratios.

The proposed algorithm matches centralized strategies in performance.

Abstract

Motivated by packet routing in computer networks, online queuing systems are composed of queues receiving packets at different rates. Repeatedly, they send packets to servers, each of them treating only at most one packet at a time. In the centralized case, the number of accumulated packets remains bounded (i.e., the system is \textit{stable}) as long as the ratio between service rates and arrival rates is larger than $1$. In the decentralized case, individual no-regret strategies ensures stability when this ratio is larger than $2$. Yet, myopically minimizing regret disregards the long term effects due to the carryover of packets to further rounds. On the other hand, minimizing long term costs leads to stable Nash equilibria as soon as the ratio exceeds $\frac{e}{e-1}$. Stability with decentralized learning strategies with a ratio below $2$ was a major remaining question. We first argue that for ratios up to $2$, cooperation is required for stability of learning strategies, as selfish minimization of policy regret, a \textit{patient} notion of regret, might indeed still be unstable in this case. We therefore consider cooperative queues and propose the first learning decentralized algorithm guaranteeing stability of the system as long as the ratio of rates is larger than $1$, thus reaching performances comparable to centralized strategies.