Learning and balancing unknown loads in large-scale systems

TL;DR

This paper introduces a learning-based control scheme for large-scale server systems with unknown loads, proving its effectiveness in balancing loads under various service time distributions without relying on traditional fluid limit analysis.

Contribution

It develops a novel proof technique to analyze load balancing and learning in large systems, handling rapid occupancy fluctuations and Coxian service times without fluid limits.

Findings

The learning scheme achieves equilibrium with balanced load distribution.

It works effectively under both exponential and Coxian service time distributions.

The approach handles rapid occupancy excursions and system variations.

Abstract

Consider a system of identical server pools where tasks with exponentially distributed service times arrive as a time-inhomogenenous Poisson process. An admission threshold is used in an inner control loop to assign incoming tasks to server pools while, in an outer control loop, a learning scheme adjusts this threshold over time to keep it aligned with the unknown offered load of the system. In a many-server regime, we prove that the learning scheme reaches an equilibrium along intervals of time where the normalized offered load per server pool is suitably bounded, and that this results in a balanced distribution of the load. Furthermore, we establish a similar result when tasks with Coxian distributed service times arrive at a constant rate and the threshold is adjusted using only the total number of tasks in the system. The novel proof technique developed in this paper, which differs from a traditional fluid limit analysis, allows to handle rapid variations of the first learning scheme, triggered by excursions of the occupancy process that have vanishing size. Moreover, our approach allows to characterize the asymptotic behavior of the system with Coxian distributed service times without relying on a fluid limit of a detailed state descriptor.