A strong law for the rate of growth of long latency periods in cloud computing service

TL;DR

This paper investigates the growth rate of long latency periods in cloud computing services, providing a mathematical framework for service guarantees and analyzing workload behavior as customer demand increases.

Contribution

It introduces a new metric for prolonged latency in cloud services and offers a large-deviation principle for workload processes, aiding in resource allocation strategies.

Findings

Characterizes capacity needed to prevent long latency durations

Provides a large-deviation principle for workload collections

Offers insights for service guarantee provisioning

Abstract

Cloud-computing shares a common pool of resources across customers at a scale that is orders of magnitude larger than traditional multi-user systems. Constituent physical compute servers are allocated multiple "virtual machines" (VM) to serve simultaneously. Each VM user should ideally be unaffected by others' demand. Naturally, this environment produces new challenges for the service providers in meeting customer expectations while extracting an efficient utilization from server resources. We study a new cloud service metric that measures prolonged latency or delay suffered by customers. We model the workload process of a cloud server and analyze the process as the customer population grows. The capacity required to ensure that average workload does not exceed a threshold over long segments is characterized. This can be used by cloud operators to provide service guarantees on avoiding long durations of latency. As part of the analysis, we provide a uniform large-deviation principle for collections of random variables that is of independent interest.