Mean field optimal Core Allocation across Malleable jobs

TL;DR

This paper addresses the optimal core allocation for malleable jobs in data centers, deriving new policies that are asymptotically optimal and applicable to diverse job classes with concave speedup functions.

Contribution

It introduces the CAM problem, solves it in a general setting with multiple job classes, and proposes two mean field optimal policies, FW-CAM and WHAM, with theoretical guarantees.

Findings

WHAM is asymptotically optimal for diverse job classes.

In the mean field regime, job sizes are irrelevant for optimal policy.

Proposed policies outperform previous methods in the general setting.

Abstract

Modern data centers and cloud computing clusters are increasingly running workloads composed of malleable jobs. A malleable job can be parallelized across any number of cores, yet the job typically exhibits diminishing marginal returns for each additional core on which it runs. This can be seen in the concavity of a job's speedup function, which describes the job's processing speed as a function of the number of cores on which it runs. Given the prevalence of malleable jobs, several theoretical works have posed the problem of how to allocate a fixed number of cores across a stream of arriving malleable jobs so as to minimize the mean response time across jobs. We refer to this as the Core Allocation to Malleable jobs (CAM) problem. We solve the CAM problem under a highly general setting, allowing for multiple job classes, each with an arbitrary concave speedup function and holding costs (weight). Furthermore, we allow for generally distributed inter-arrival times and job sizes. We analyze the CAM problem in the mean field asymptotic regime and derive two distinct mean field optimal policies, FW-CAM and WHAM. FW-CAM is interesting because it demonstrates a new intuition: in the mean field regime, job sizes are not relevant in finding an optimal policy. WHAM (Whittle Allocation for Malleable jobs) is interesting because it is asymptotically optimal and also serves as a good heuristic even outside of the asymptotic regime. Notably, none of the policies previously proposed in the literature are mean field optimal when jobs may follow different speedup functions.