Optimal Resource Allocation for Elastic and Inelastic Jobs

TL;DR

This paper studies how to optimally allocate servers in data centers to elastic and inelastic jobs to minimize response time, revealing that prioritizing inelastic jobs is optimal when they are generally larger, with analysis of both policies.

Contribution

It introduces and analyzes the Inelastic-First policy for resource allocation in heterogeneous workloads, providing the first mean response time analysis for both Elastic-First and Inelastic-First policies.

Findings

Inelastic-First policy is optimal when elastic jobs are smaller on average.

Elastic-First can outperform Inelastic-First in certain cases.

The paper provides a novel sample path argument and Markov chain analysis for response times.

Abstract

Modern data centers are tasked with processing heterogeneous workloads consisting of various classes of jobs. These classes differ in their arrival rates, size distributions, and job parallelizability. With respect to paralellizability, some jobs are elastic, meaning they can parallelize linearly across many servers. Other jobs are inelastic, meaning they can only run on a single server. Although job classes can differ drastically, they are typically forced to share a single cluster. When sharing a cluster among heterogeneous jobs, one must decide how to allocate servers to each job at every moment in time. In this paper, we design and analyze allocation policies which aim to minimize the mean response time across jobs, where a job's response time is the time from when it arrives until it completes. We model this problem in a stochastic setting where each job may be elastic or inelastic. Job sizes are drawn from exponential distributions, but are unknown to the system. We show that, in the common case where elastic jobs are larger on average than inelastic jobs, the optimal allocation policy is Inelastic-First, giving inelastic jobs preemptive priority over elastic jobs. We obtain this result by introducing a novel sample path argument. We also show that there exist cases where Elastic-First (giving priority to elastic jobs) performs better than Inelastic-First. We then provide the first analysis of mean response time under both Elastic-First and Inelastic-First by leveraging recent techniques for solving high-dimensional Markov chains.