Mirage: Towards Low-interruption Services on Batch GPU Clusters with Reinforcement Learning

TL;DR

Mirage is a reinforcement learning-based resource provisioner designed to reduce job interruptions on GPU clusters, significantly improving the quality of service for deep learning workloads by proactively managing resources.

Contribution

This paper introduces Mirage, a novel RL-based system that proactively reduces job interruptions on GPU clusters, enhancing productivity and QoS for deep learning tasks.

Findings

Reduces job interruptions by up to 100%

Safeguards up to 76% of jobs with zero interruption

Effective across different load levels

Abstract

Accommodating long-running deep learning (DL) training and inference jobs is challenging on GPU clusters that use traditional batch schedulers, such as Slurm. Given fixed wall clock time limits, DL researchers usually need to run a sequence of batch jobs and experience long interruptions on overloaded machines. Such interruptions significantly lower the research productivity and QoS for services that are deployed in production. To mitigate the issues from interruption, we investigate a set of statistical learning and reinforcement learning (RL) techniques, including random forest, xgboost, Deep Q-Network, and policy gradient to design a proactive provisioner using production job traces from three GPU clusters. We follow the standard machine learning practice by partitioning each job trace into training and validation subsets, then train each model using the training subset and evaluate the generality using the validation subset. We introduce Mirage, a Slurm-compatible resource provisioner that integrates the candidate RL methods. Our experiments show that the Mirage can reduce the interruption by 17-100% and safeguard 23%-76% of jobs with zero interruption across varying load levels on the three clusters.