A Cost Effective Reliability Aware Scheduler for Task Graphs in Multi-Cloud System

TL;DR

This paper introduces a cost-effective, reliability-aware scheduling algorithm for task graphs in multi-cloud systems, optimizing workflow execution considering cost, time, and reliability.

Contribution

It presents a novel heuristic scheduler that accounts for practical factors like pricing, discounts, and reliability, outperforming existing methods.

Findings

Outperforms state-of-the-art in cost by up to 12%

Reduces makespan by up to 11%

Improves reliability by approximately 1.1%

Abstract

Many scientific workflows can be represented by a Directed Acyclic Graph (DAG) where each node represents a task, and there will be a directed edge between two tasks if and only if there is a dependency relationship between the two i.e. the second one can not be started unless the first one is finished. Due to the increasing computational requirements of these workflows, they are deployed on cloud computing systems. Scheduling of workflows on such systems to achieve certain goals(e.g. minimization of makespan, cost, or maximization of reliability, etc.) remains an active area of research. In this paper, we propose a scheduling algorithm for allocating the nodes of our task graph in a heterogeneous multi-cloud system. The proposed scheduler considers many practical concerns such as pricing mechanisms, discounting schemes, and reliability analysis for task execution. This is a list-based heuristic that allocates tasks based on the expected times for which VMs need to be rented for them. We have analyzed the proposed approach to understand its time requirement. We perform a large number of experiments with real-world workflows: FFT, Ligo, Epigenomics, and Random workflows and observe that the proposed scheduler outperforms the state-of-art approaches up to 12%, 11%, and 1.1% in terms of cost, makespan, and reliability, respectively.