Using Genetic Algorithms to Benchmark the Cloud

TL;DR

This paper introduces a novel method using Genetic Algorithms to benchmark cloud PaaS providers, revealing performance differences and client dependency, thus enhancing cloud service evaluation.

Contribution

The paper presents a new approach applying GAs to quantify PaaS performance, leveraging cloud elasticity for more nuanced benchmarking.

Findings

Significant performance differences among PaaS vendors.

Client computer class affects PaaS performance.

GA effectively assesses service levels and vendor repeatability.

Abstract

This paper presents a novel application of Genetic Algorithms(GAs) to quantify the performance of Platform as a Service (PaaS), a cloud service model that plays a critical role in both industry and academia. While Cloud benchmarks are not new, in this novel concept, the authors use a GA to take advantage of the elasticity in Cloud services in a graceful manner that was not previously possible. Using Google App Engine, Heroku, and Python Anywhere with three distinct classes of client computers running our GA codebase, we quantified the completion time for application of the GA to search for the parameters of controllers for dynamical systems. Our results show statistically significant differences in PaaS performance by vendor, and also that the performance of the PaaS performance is dependent upon the client that uses it. Results also show the effectiveness of our GA in determining the level of service of PaaS providers, and for determining if the level of service of one PaaS vendor is repeatable with another. Such a concept could then increase the appeal of PaaS Cloud services by making them more financially appealing.