Scalability of VM Provisioning Systems

TL;DR

This paper evaluates the startup performance of three major cloud management frameworks in HPC environments, revealing significant differences and providing a methodology for measurement and optimization.

Contribution

It introduces a methodology for measuring VM startup performance and compares three cloud frameworks, highlighting their suitability for HPC workloads.

Findings

Eucalyptus has the fastest startup time.

OpenNebula has the slowest startup time.

Networking delays in cloud-init cause most startup overhead.

Abstract

Virtual machines and virtualized hardware have been around for over half a century. The commoditization of the x86 platform and its rapidly growing hardware capabilities have led to recent exponential growth in the use of virtualization both in the enterprise and high performance computing (HPC). The startup time of a virtualized environment is a key performance metric for high performance computing in which the runtime of any individual task is typically much shorter than the lifetime of a virtualized service in an enterprise context. In this paper, a methodology for accurately measuring the startup performance on an HPC system is described. The startup performance overhead of three of the most mature, widely deployed cloud management frameworks (OpenStack, OpenNebula, and Eucalyptus) is measured to determine their suitability for workloads typically seen in an HPC environment. A 10x performance difference is observed between the fastest (Eucalyptus) and the slowest (OpenNebula) framework. This time difference is primarily due to delays in waiting on networking in the cloud-init portion of the startup. The methodology and measurements presented should facilitate the optimization of startup across a variety of virtualization environments.