Resource allocation and routing in parallel multi-server queues with abandonments for cloud profit maximization

TL;DR

This paper develops and tests low-complexity heuristics for resource allocation and routing in cloud services with job deadlines, aiming to maximize profit despite the complexity of optimal policy computation.

Contribution

It introduces and compares static and dynamic heuristic policies for profit maximization in multi-server cloud queues with abandonments, addressing computational intractability.

Findings

Dynamic index policies outperform static policies in profit.

Heuristics show robust performance across various scenarios.

Insights into policy strengths and weaknesses are provided.

Abstract

This paper considers a Markov decision model for profit maximization of a cloud computing service provider catering to customers submitting jobs with firm real-time random deadlines. Customers are charged on a per-job basis, receiving a full refund if deadlines are missed. The service provider leases computing resources from an infrastructure provider in a two-tier scheme: long-term leasing of basic infrastructure, consisting of heterogeneous parallel service nodes, each modeled as a multi-server queue, and short-term leasing of external servers. Given the intractability of computing an optimal dynamic resource allocation and job routing policy, maximizing the long-run average profit rate, the paper addresses the design, implementation and testing of low-complexity heuristics. The policies considered are a static policy given by an optimal Bernoulli splitting, and three dynamic index policies based on different index definitions: individually optimal (IO), policy improvement (PI) and restless bandit (RB) indices. The paper shows how to implement efficiently each such policy, and presents a comprehensive empirical comparison, drawing qualitative insights on their strengths and weaknesses, and benchmarking their performance in an extensive study.