Quantifying the Performance Gap for Simple Versus Optimal Dynamic Server Allocation Policies

TL;DR

This paper compares simple and optimal dynamic server allocation policies in cloud computing, quantifying their performance differences and exploring benefits of state-dependent routing for cost and delay optimization.

Contribution

It introduces analytic models for simple policies, develops semi-Markov decision models for optimal policies, and assesses performance gaps and routing benefits in multi-site systems.

Findings

Simple policies are effective but less optimal than complex ones.

Optimal policies significantly improve performance metrics.

State-dependent routing offers notable benefits in multi-site cloud systems.

Abstract

Cloud computing enables the dynamic provisioning of server resources. To exploit this opportunity, a policy is needed for dynamically allocating (and deallocating) servers in response to the current load conditions. In this paper we describe several simple policies for dynamic server allocation and develop analytic models for their analysis. We also design semi-Markov decision models that enable determination of the performance achieved with optimal policies, allowing us to quantify the performance gap between simple, easily implemented policies, and optimal policies. Finally, we apply our models to study the potential performance benefits of state-dependent routing in multi-site systems when using dynamic server allocation at each site. Insights from our results are valuable to service providers wanting to balance cloud service costs and delays.