Optimizing CPU Cache Utilization in Cloud VMs with Accurate Cache Abstraction

TL;DR

This paper introduces CacheX, a novel method for accurately probing cache details within cloud VMs using eviction sets, enabling better cache management and optimization without hardware or hypervisor modifications.

Contribution

CacheX provides a new fine-grained cache abstraction technique for cloud VMs, facilitating cache-aware scheduling and page management to improve cache utilization.

Findings

CacheX effectively probes cache details without hardware support.

CacheX improves cache utilization in cloud VMs.

Enhanced cache management techniques lead to performance gains.

Abstract

This paper shows that cache-based optimizations are often ineffective in cloud virtual machines (VMs) due to limited visibility into and control over provisioned caches. In public clouds, CPU caches can be partitioned or shared among VMs, but a VM is unaware of cache provisioning details. Moreover, a VM cannot influence cache usage via page placement policies, as memory-to-cache mappings are hidden. The paper proposes a novel solution, CacheX, which probes accurate and fine-grained cache abstraction within VMs using eviction sets without requiring hardware or hypervisor support, and showcases the utility of the probed information with two new techniques: LLC contention-aware task scheduling and virtual color-aware page cache management. Our evaluation of CacheX's implementation in x86 Linux kernel demonstrates that it can effectively improve cache utilization for various workloads in public cloud VMs.