Load balancing with heterogeneous schedulers

TL;DR

This paper develops a load balancing strategy for heterogeneous servers with limited processor sharing disciplines, using Whittle's index to optimize request routing and outperform standard heuristics.

Contribution

It introduces a Whittle's index-based load balancing approach for servers with heterogeneous limited processor sharing disciplines, providing a closed-form index and demonstrating superior performance.

Findings

Whittle's index policy depends on the server's scheduling discipline.

Whittle's index policy outperforms JSQ, JSEW, and RSA.

Performance varies with different cost criteria.

Abstract

Load balancing is a common approach in web server farms or inventory routing problems. An important issue in such systems is to determine the server to which an incoming request should be routed to optimize a given performance criteria. In this paper, we assume the server's scheduling disciplines to be heterogeneous. More precisely, a server implements a scheduling discipline which belongs to the class of limited processor sharing (LPS-$d$) scheduling disciplines. Under LPS-$d$, up to $d$ jobs can be served simultaneously, and hence, includes as special cases First Come First Served ($d=1$) and Processor Sharing ($d=\infty$). In order to obtain efficient heuristics, we model the above load-balancing framework as a multi-armed restless bandit problem. Using the relaxation technique, as first developed in the seminal work of Whittle, we derive Whittle's index policy for general cost functions and obtain a closed-form expression for Whittle's index in terms of the steady-state distribution. Through numerical computations, we investigate the performance of Whittle's index with two different performance criteria: linear cost criterion and a cost criterion that depends on the first and second moment of the throughput. Our results show that \emph{(i)} the structure of Whittle's index policy can strongly depend on the scheduling discipline implemented in the server, i.e., on $d$, and that \emph{(ii)} Whittle's index policy significantly outperforms standard dispatching rules such as Join the Shortest Queue (JSQ), Join the Shortest Expected Workload (JSEW), and Random Server allocation (RSA).