Vmem: A Lightweight Hot-Upgradable Memory Management for In-production Cloud Environment

TL;DR

Vmem is a lightweight, upgradable memory management system designed for cloud environments, significantly improving memory utilization, stability, and performance with minimal overhead and supporting online upgrades.

Contribution

It introduces the first architecture supporting online upgrades for cloud memory management, enhancing flexibility, stability, and efficiency in production environments.

Findings

Increases sellable memory rate by about 2%

Achieves over 3x faster VM boot times

Improves network performance by about 10%

Abstract

Traditional memory management suffers from metadata overhead, architectural complexity, and stability degradation, problems intensified in cloud environments. Existing software/hardware optimizations are insufficient for cloud computing's dual demands of flexibility and low overhead. This paper presents Vmem, a memory management architecture for in-production cloud environments that enables flexible, efficient cloud server memory utilization through lightweight reserved memory management. Vmem is the first such architecture to support online upgrades, meeting cloud requirements for high stability and rapid iterative evolution. Experiments show Vmem increases sellable memory rate by about 2%, delivers extreme elasticity and performance, achieves over 3x faster boot time for VFIO-based virtual machines (VMs), and improves network performance by about 10% for DPU-accelerated VMs. Vmem has been deployed at large scale for seven years, demonstrating efficiency and stability on over 300,000 cloud servers supporting hundreds of millions of VMs.