Hybrid Learning and Optimization-Based Dynamic Scheduling for DL Workloads on Heterogeneous GPU Clusters

TL;DR

This paper introduces RLTune, a reinforcement learning-based dynamic scheduling framework for heterogeneous GPU clusters, significantly improving resource utilization and reducing delays for large-scale deep learning workloads without needing per-job profiling.

Contribution

RLTune is an application-agnostic RL-based scheduler that combines RL prioritization with MILP-based job mapping, enabling scalable and efficient DL workload management on heterogeneous GPU clusters.

Findings

GPU utilization increased by up to 20%

Queueing delay reduced by up to 81%

Job completion time shortened by up to 70%

Abstract

Modern cloud platforms increasingly host large-scale deep learning (DL) workloads, demanding high-throughput, low-latency GPU scheduling. However, the growing heterogeneity of GPU clusters and limited visibility into application characteristics pose major challenges for existing schedulers, which often rely on offline profiling or application-specific assumptions. We present RLTune, an application-agnostic reinforcement learning (RL)-based scheduling framework that dynamically prioritizes and allocates DL jobs on heterogeneous GPU clusters. RLTune integrates RL-driven prioritization with MILP-based job-to-node mapping to optimize system-wide objectives such as job completion time (JCT), queueing delay, and resource utilization. Trained on large-scale production traces from Microsoft Philly, Helios, and Alibaba, RLTune improves GPU utilization by up to 20%, reduces queueing delay by up to 81%, and shortens JCT by as much as 70 percent. Unlike prior approaches, RLTune generalizes across diverse workloads without requiring per-job profiling, making it practical for cloud providers to deploy at scale for more efficient, fair, and sustainable DL workload management.