Reinforcement Learning based Workflow Scheduling in Cloud and Edge Computing Environments: A Taxonomy, Review and Future Directions

TL;DR

This paper reviews the application of Deep Reinforcement Learning for workflow scheduling in cloud and edge computing, analyzing challenges, proposing a taxonomy, and outlining future research directions.

Contribution

It provides a comprehensive taxonomy of DRL-based workflow scheduling and maps existing works to identify strengths, weaknesses, and future research opportunities.

Findings

Identified key challenges like multi-objectivity and partial observability.

Mapped existing solutions within the proposed taxonomy.

Outlined future directions for DRL in workflow scheduling.

Abstract

Deep Reinforcement Learning (DRL) techniques have been successfully applied for solving complex decision-making and control tasks in multiple fields including robotics, autonomous driving, healthcare and natural language processing. The ability of DRL agents to learn from experience and utilize real-time data for making decisions makes it an ideal candidate for dealing with the complexities associated with the problem of workflow scheduling in highly dynamic cloud and edge computing environments. Despite the benefits of DRL, there are multiple challenges associated with the application of DRL techniques including multi-objectivity, curse of dimensionality, partial observability and multi-agent coordination. In this paper, we comprehensively analyze the challenges and opportunities associated with the design and implementation of DRL oriented solutions for workflow scheduling in cloud and edge computing environments. Based on the identified characteristics, we propose a taxonomy of workflow scheduling with DRL. We map reviewed works with respect to the taxonomy to identify their strengths and weaknesses. Based on taxonomy driven analysis, we propose novel future research directions for the field.