Monetary Cost Optimizations for Hosting Workflow-as-a-Service in IaaS Clouds

TL;DR

This paper introduces Dyna, a probabilistic scheduling framework for WaaS in IaaS clouds that minimizes costs while providing probabilistic deadline guarantees, accounting for cloud performance and price variability.

Contribution

It presents a novel probabilistic scheduling approach with A*-based instance configuration and spot instance refinement, addressing dynamic cloud environments.

Findings

Accurately predicts probabilistic deadline satisfaction.

Reduces monetary costs compared to existing methods.

Effective in real-world scientific workflows.

Abstract

Recently, we have witnessed workflows from science and other data-intensive applications emerging on Infrastructure-asa-Service (IaaS) clouds, and many workflow service providers offering workflow as a service (WaaS). The major concern of WaaS providers is to minimize the monetary cost of executing workflows in the IaaS cloud. While there have been previous studies on this concern, most of them assume static task execution time and static pricing scheme, and have the QoS notion of satisfying a deterministic deadline. However, cloud environment is dynamic, with performance dynamics caused by the interference from concurrent executions and price dynamics like spot prices offered by Amazon EC2. Therefore, we argue that WaaS providers should have the notion of offering probabilistic performance guarantees for individual workflows on IaaS clouds. We develop a probabilistic scheduling framework called Dyna to minimize the monetary cost while offering probabilistic deadline guarantees. The framework includes an A*-based instance configuration method for performance dynamics, and a hybrid instance configuration refinement for utilizing spot instances. Experimental results with three real-world scientific workflow applications on Amazon EC2 demonstrate (1) the accuracy of our framework on satisfying the probabilistic deadline guarantees required by the users; (2) the effectiveness of our framework on reducing monetary cost in comparison with the existing approaches.