Coach: Exploiting Temporal Patterns for All-Resource Oversubscription in Cloud Platforms

TL;DR

This paper introduces Coach, a system that leverages temporal resource usage patterns and predictive scheduling to safely oversubscribe all resources in cloud platforms, significantly increasing VM density.

Contribution

Coach is the first system to exploit temporal patterns for holistic resource oversubscription across all VM resources in cloud environments.

Findings

Enables hosting up to 26% more VMs with minimal performance impact.

Effectively manages memory contention through a novel partitioning approach.

Utilizes long-term predictions for efficient VM scheduling.

Abstract

Cloud platforms remain underutilized despite multiple proposals to improve their utilization (e.g., disaggregation, harvesting, and oversubscription). Our characterization of the resource utilization of virtual machines (VMs) in Azure reveals that, while CPU is the main underutilized resource, we need to provide a solution to manage all resources holistically. We also observe that many VMs exhibit complementary temporal patterns, which can be leveraged to improve the oversubscription of underutilized resources. Based on these insights, we propose Coach: a system that exploits temporal patterns for all-resource oversubscription in cloud platforms. Coach uses long-term predictions and an efficient VM scheduling policy to exploit temporally complementary patterns. We introduce a new general-purpose VM type, called CoachVM, where we partition each resource allocation into a guaranteed and an oversubscribed portion. Coach monitors the oversubscribed resources to detect contention and mitigate any potential performance degradation. We focus on memory management, which is particularly challenging due to memory's sensitivity to contention and the overhead required to reassign it between CoachVMs. Our experiments show that Coach enables platforms to host up to ~26% more VMs with minimal performance degradation.