Analysis of Reinforcement Learning for determining task replication in workflows

TL;DR

This paper explores using reinforcement learning to optimize task replication in volunteer computing workflows, aiming to improve completion times while reducing energy consumption and system load.

Contribution

It introduces a reinforcement learning approach to dynamically determine the optimal number of task replicas, balancing efficiency and resource usage.

Findings

Saved 34% energy consumption with RL

Achieved only 4% decrease in workflow completion within overhead bounds

Demonstrated effectiveness through simulation

Abstract

Executing workflows on volunteer computing resources where individual tasks may be forced to relinquish their resource for the resource's primary use leads to unpredictability and often significantly increases execution time. Task replication is one approach that can ameliorate this challenge. This comes at the expense of a potentially significant increase in system load and energy consumption. We propose the use of Reinforcement Learning (RL) such that a system may `learn' the `best' number of replicas to run to increase the number of workflows which complete promptly whilst minimising the additional workload on the system when replicas are not beneficial. We show, through simulation, that we can save 34% of the energy consumption using RL compared to a fixed number of replicas with only a 4% decrease in workflows achieving a pre-defined overhead bound.