Towards VM Rescheduling Optimization Through Deep Reinforcement Learning

TL;DR

This paper introduces VM2RL, a deep reinforcement learning approach for VM rescheduling in data centers, addressing scalability and performance issues of existing methods by optimizing rescheduling decisions with a customizable, efficient system.

Contribution

The paper presents VM2RL, a novel RL-based system for VM rescheduling that handles diverse constraints and workloads, achieving near-optimal performance with significantly reduced inference time.

Findings

VM2RL achieves near-optimal rescheduling performance.

The system runs in seconds, outperforming existing methods in speed.

Open-source code and datasets support reproducibility.

Abstract

Modern industry-scale data centers need to manage a large number of virtual machines (VMs). Due to the continual creation and release of VMs, many small resource fragments are scattered across physical machines (PMs). To handle these fragments, data centers periodically reschedule some VMs to alternative PMs, a practice commonly referred to as VM rescheduling. Despite the increasing importance of VM rescheduling as data centers grow in size, the problem remains understudied. We first show that, unlike most combinatorial optimization tasks, the inference time of VM rescheduling algorithms significantly influences their performance, due to dynamic VM state changes during this period. This causes existing methods to scale poorly. Therefore, we develop a reinforcement learning system for VM rescheduling, VM2RL, which incorporates a set of customized techniques, such as a two-stage framework that accommodates diverse constraints and workload conditions, a feature extraction module that captures relational information specific to rescheduling, as well as a risk-seeking evaluation enabling users to optimize the trade-off between latency and accuracy. We conduct extensive experiments with data from an industry-scale data center. Our results show that VM2RL can achieve a performance comparable to the optimal solution but with a running time of seconds. Code and datasets are open-sourced: https://github.com/zhykoties/VMR2L_eurosys, https://drive.google.com/drive/folders/1PfRo1cVwuhH30XhsE2Np3xqJn2GpX5qy.