Scheduling Algorithms for Efficient Execution of Stream Workflow Applications in Multicloud Environments

TL;DR

This paper proposes two multicloud scheduling algorithms for stream workflow applications, aiming to optimize execution efficiency, meet user deadlines, and reduce costs in complex big data processing environments.

Contribution

It introduces novel multicloud scheduling and resource allocation techniques specifically designed for stream workflow applications, enhancing performance and cost-effectiveness.

Findings

Genetic algorithm effectively optimizes scheduling

Proposed methods meet user deadlines consistently

Cost reduction achieved in multicloud execution

Abstract

Big data processing applications are becoming more and more complex. They are no more monolithic in nature but instead they are composed of decoupled analytical processes in the form of a workflow. One type of such workflow applications is stream workflow application, which integrates multiple streaming big data applications to support decision making. Each analytical component of these applications runs continuously and processes data streams whose velocity will depend on several factors such as network bandwidth and processing rate of parent analytical component. As a consequence, the execution of these applications on cloud environments requires advanced scheduling techniques that adhere to end user's requirements in terms of data processing and deadline for decision making. In this paper, we propose two Multicloud scheduling and resource allocation techniques for efficient execution of stream workflow applications on Multicloud environments while adhering to workflow application and user performance requirements and reducing execution cost. Results showed that the proposed genetic algorithm is an adequate and effective for all experiments.