Scientific Workflow Applications on Amazon EC2

TL;DR

This paper evaluates the performance and cost of running scientific workflows on Amazon EC2, comparing it to traditional HPC systems, and finds that clouds can offer comparable performance with cost benefits through data storage strategies.

Contribution

It provides an empirical comparison of cloud and HPC performance for scientific workflows and analyzes cost factors, highlighting cloud viability for scientific computing.

Findings

Cloud performance is comparable to HPC with similar resources.

Storing data in the cloud reduces overall workflow costs.

Performance is reasonable given hardware resources.

Abstract

The proliferation of commercial cloud computing providers has generated significant interest in the scientific computing community. Much recent research has attempted to determine the benefits and drawbacks of cloud computing for scientific applications. Although clouds have many attractive features, such as virtualization, on-demand provisioning, and "pay as you go" usage-based pricing, it is not clear whether they are able to deliver the performance required for scientific applications at a reasonable price. In this paper we examine the performance and cost of clouds from the perspective of scientific workflow applications. We use three characteristic workflows to compare the performance of a commercial cloud with that of a typical HPC system, and we analyze the various costs associated with running those workflows in the cloud. We find that the performance of clouds is not unreasonable given the hardware resources provided, and that performance comparable to HPC systems can be achieved given similar resources. We also find that the cost of running workflows on a commercial cloud can be reduced by storing data in the cloud rather than transferring it from outside.