Improved Load Balancing in Large Scale Systems using Attained Service Time Reporting

TL;DR

This paper proposes load balancing policies in large-scale systems that utilize attained service time information alongside queue length to improve job response times, especially under high variability in job sizes.

Contribution

It introduces and analyzes novel load balancing policies that incorporate attained service time, demonstrating their effectiveness through theoretical analysis and simulations.

Findings

Significant reduction in waiting times using attained service time information.

Policies using only attained service time outperform traditional queue length-based methods.

Improved response times in moderately loaded systems without queue length data.

Abstract

Our interest lies in load balancing jobs in large scale systems consisting of multiple dispatchers and FCFS servers. In the absence of any information on job sizes, dispatchers typically use queue length information reported by the servers to assign incoming jobs. When job sizes are highly variable, using only queue length information is clearly suboptimal and performance can be improved if some indication can be provided to the dispatcher about the size of an ongoing job. In a FCFS server measuring the attained service time of the ongoing job is easy and servers can therefore report this attained service time together with the queue length when queried by a dispatcher. In this paper we propose and analyse a variety of load balancing policies that exploit both the queue length and attained service time to assign jobs, as well as policies for which only the attained service time of the job in service is used. We present a unified analysis for all these policies in a large scale system under the usual asymptotic independence assumptions. The accuracy of the proposed analysis is illustrated using simulation. We present extensive numerical experiments which clearly indicate that a significant improvement in waiting (and thus also in response) time may be achieved by using the attained service time information on top of the queue length of a server. Moreover, the policies which do not make use of the queue length still provide an improved waiting time for moderately loaded systems.