Leveraging Reinforcement Learning for Task Resource Allocation in Scientific Workflows

TL;DR

This paper introduces reinforcement learning methods to optimize resource allocation in scientific workflows, reducing wastage and improving efficiency compared to traditional and existing feedback-based approaches.

Contribution

It presents two novel RL approaches for resource allocation in scientific workflows, implemented in Nextflow, and demonstrates significant improvements over baseline methods.

Findings

Reinforcement learning approaches significantly reduce resource wastage.

Our methods decrease CPU hours compared to feedback loop baseline.

Approaches outperform default resource configurations.

Abstract

Scientific workflows are designed as directed acyclic graphs (DAGs) and consist of multiple dependent task definitions. They are executed over a large amount of data, often resulting in thousands of tasks with heterogeneous compute requirements and long runtimes, even on cluster infrastructures. In order to optimize the workflow performance, enough resources, e.g., CPU and memory, need to be provisioned for the respective tasks. Typically, workflow systems rely on user resource estimates which are known to be highly error-prone and can result in over- or underprovisioning. While resource overprovisioning leads to high resource wastage, underprovisioning can result in long runtimes or even failed tasks. In this paper, we propose two different reinforcement learning approaches based on gradient bandits and Q-learning, respectively, in order to minimize resource wastage by selecting suitable CPU and memory allocations. We provide a prototypical implementation in the well-known scientific workflow management system Nextflow, evaluate our approaches with five workflows, and compare them against the default resource configurations and a state-of-the-art feedback loop baseline. The evaluation yields that our reinforcement learning approaches significantly reduce resource wastage compared to the default configuration. Further, our approaches also reduce the allocated CPU hours compared to the state-of-the-art feedback loop by 6.79% and 24.53%.