Cloud Benchmarking for Performance

TL;DR

This paper presents a six-step benchmarking methodology that helps users select optimal cloud VMs based on application-specific performance weights, validated through case studies demonstrating improved performance.

Contribution

It introduces a novel, customizable benchmarking approach that ranks cloud VMs according to user-defined importance weights for different resource groups.

Findings

The methodology effectively identifies top-performing VMs for diverse workloads.

Case studies confirm the approach's ability to maximize application performance.

The ranking aligns with empirical performance measurements.

Abstract

How can applications be deployed on the cloud to achieve maximum performance? This question has become significant and challenging with the availability of a wide variety of Virtual Machines (VMs) with different performance capabilities in the cloud. The above question is addressed by proposing a six step benchmarking methodology in which a user provides a set of four weights that indicate how important each of the following groups: memory, processor, computation and storage are to the application that needs to be executed on the cloud. The weights along with cloud benchmarking data are used to generate a ranking of VMs that can maximise performance of the application. The rankings are validated through an empirical analysis using two case study applications; the first is a financial risk application and the second is a molecular dynamics simulation, which are both representative of workloads that can benefit from execution on the cloud. Both case studies validate the feasibility of the methodology and highlight that maximum performance can be achieved on the cloud by selecting the top ranked VMs produced by the methodology.