Flexible Swapping for the Cloud

TL;DR

This paper presents a customizable userspace memory management framework for VMs in cloud data centers that improves memory overcommit efficiency and performance over existing Linux-based solutions.

Contribution

It introduces a flexible, easy-to-deploy userspace API for VM memory management that supports custom policies, huge page swapping, and zero-copy I/O virtualization.

Findings

Outperforms Linux kernel baseline by up to 25% in efficiency.

Saves up to 10% additional memory with custom reclaimers and prefetchers.

Improves performance by 30% in limited memory scenarios.

Abstract

Memory has become the primary cost driver in cloud data centers. Yet, a significant portion of memory allocated to VMs in public clouds remains unused. To optimize this resource, "cold" memory can be reclaimed from VMs and stored on slower storage or compressed, enabling memory overcommit. Current overcommit systems rely on general-purpose OS swap mechanisms, which are not optimized for virtualized workloads, leading to missed memory-saving opportunities and ineffective use of optimizations like prefetchers. This paper introduces a userspace memory management framework designed for VMs. It enables custom policies that have full control over the virtual machines' memory using a simple userspace API, supports huge page-based swapping to satisfy VM performance requirements, is easy to deploy by leveraging Linux/KVM, and supports zero-copy I/O virtualization with shared VM memory. Our evaluation demonstrates that an overcommit system based on our framework outperforms the state-of-the-art solutions on both micro-benchmarks and commonly used cloud workloads. Specifically our implementation outperforms the Linux Kernel baseline implementation by up to 25% while saving a similar amount of memory. We also demonstrate the benefits of custom policies by implementing workload-specific reclaimers and prefetchers that save $10\%$ additional memory, improve performance in a limited memory scenario by 30% over the Linux baseline, and recover faster from hard limit releases.