Cost-Aware Dynamic Cloud Workflow Scheduling using Self-Attention and Evolutionary Reinforcement Learning

TL;DR

This paper introduces a cost-aware cloud workflow scheduling method using a self-attention neural network and evolutionary reinforcement learning, significantly improving cost efficiency and decision quality over existing algorithms.

Contribution

It proposes a novel self-attention policy network for cloud scheduling and an ERL training system, enhancing global VM information utilization and scheduling effectiveness.

Findings

Outperforms state-of-the-art algorithms on benchmark problems

Effectively captures global VM information for scheduling decisions

Reduces total costs including SLA penalties and VM fees

Abstract

The Cost-aware Dynamic Multi-Workflow Scheduling (CDMWS) in the cloud is a kind of cloud workflow management problem, which aims to assign virtual machine (VM) instances to execute tasks in workflows so as to minimize the total costs, including both the penalties for violating Service Level Agreement (SLA) and the VM rental fees. Powered by deep neural networks, Reinforcement Learning (RL) methods can construct effective scheduling policies for solving CDMWS problems. Traditional policy networks in RL often use basic feedforward architectures to separately determine the suitability of assigning any VM instances, without considering all VMs simultaneously to learn their global information. This paper proposes a novel self-attention policy network for cloud workflow scheduling (SPN-CWS) that captures global information from all VMs. We also develop an Evolution Strategy-based RL (ERL) system to train SPN-CWS reliably and effectively. The trained SPN-CWS can effectively process all candidate VM instances simultaneously to identify the most suitable VM instance to execute every workflow task. Comprehensive experiments show that our method can noticeably outperform several state-of-the-art algorithms on multiple benchmark CDMWS problems.