Dynamic load balancing for cloud systems under heterogeneous setup delays

TL;DR

This paper introduces two load balancing policies for heterogeneous cloud systems with setup delays, optimizing for minimal setup time and queueing delay through fluid models and convex optimization.

Contribution

It proposes a myopic policy and a proximal optimization-based policy for load balancing, with theoretical convergence proofs and improved delay management.

Findings

The myopic policy converges to a setup time-minimizing equilibrium.

The proximal optimization policy achieves an equilibrium with no queueing delay.

Simulation results validate the effectiveness of both policies.

Abstract

We consider a distributed cloud service deployed at a set of distinct server pools. Arriving jobs are classified into heterogeneous types, in accordance with their setup times which are differentiated at each of the pools. A dispatcher for each job type controls the balance of load between pools, based on decentralized feedback. The system of rates and queues is modeled by a fluid differential equation system, and analyzed via convex optimization. A first, myopic policy is proposed, based on task delay-to-service. Under a simplified dynamic fluid queue model, we prove global convergence to an equilibrium point which minimizes the mean setup time; however queueing delays are incurred with this method. A second proposal is then developed based on proximal optimization, which explicitly models the setup queue and is proved to reach an optimal equilibrium, devoid of queueing delay. Results are demonstrated through a simulation example.