Dominant Resource Fairness in Cloud Computing Systems with Heterogeneous Servers

TL;DR

This paper introduces DRFH, a multi-resource allocation mechanism for heterogeneous cloud servers, ensuring fairness and incentivizing truthful demand reporting, leading to improved resource utilization and faster job completion.

Contribution

The paper generalizes Dominant Resource Fairness to heterogeneous servers and designs a practical heuristic for real-world implementation.

Findings

DRFH outperforms traditional schedulers in resource utilization.

DRFH reduces job completion times significantly.

The mechanism incentivizes truthful resource demand reporting.

Abstract

We study the multi-resource allocation problem in cloud computing systems where the resource pool is constructed from a large number of heterogeneous servers, representing different points in the configuration space of resources such as processing, memory, and storage. We design a multi-resource allocation mechanism, called DRFH, that generalizes the notion of Dominant Resource Fairness (DRF) from a single server to multiple heterogeneous servers. DRFH provides a number of highly desirable properties. With DRFH, no user prefers the allocation of another user; no one can improve its allocation without decreasing that of the others; and more importantly, no user has an incentive to lie about its resource demand. As a direct application, we design a simple heuristic that implements DRFH in real-world systems. Large-scale simulations driven by Google cluster traces show that DRFH significantly outperforms the traditional slot-based scheduler, leading to much higher resource utilization with substantially shorter job completion times.