SLA-MORL: SLA-Aware Multi-Objective Reinforcement Learning for HPC Resource Optimization

TL;DR

SLA-MORL is an adaptive multi-objective reinforcement learning framework that optimizes HPC resource allocation for machine learning workloads, balancing training time, cost, and SLA compliance through intelligent initialization and dynamic weight adjustment.

Contribution

It introduces a novel RL-based approach with cold-start reduction and real-time priority adaptation for SLA-aware resource management in HPC environments.

Findings

Achieves 67.2% reduction in training time for deadline jobs

Reduces operational costs by 68.8% for budget workloads

Improves SLA compliance by 73.4% over static methods

Abstract

Dynamic resource allocation for machine learning workloads in cloud environments remains challenging due to competing objectives of minimizing training time and operational costs while meeting Service Level Agreement (SLA) constraints. Traditional approaches employ static resource allocation or single-objective optimization, leading to either SLA violations or resource waste. We present SLA-MORL, an adaptive multi-objective reinforcement learning framework that intelligently allocates GPU and CPU resources based on user-defined preferences (time, cost, or balanced) while ensuring SLA compliance. Our approach introduces two key innovations: (1) intelligent initialization through historical learning or efficient baseline runs that eliminates cold-start problems, reducing initial exploration overhead by 60%, and (2) dynamic weight adaptation that automatically adjusts optimization priorities based on real-time SLA violation severity, creating a self-correcting system. SLA-MORL constructs a 21-dimensional state representation capturing resource utilization, training progress, and SLA compliance, enabling an actor-critic network to make informed allocation decisions across 9 possible actions. Extensive evaluation on 13 diverse ML workloads using production HPC infrastructure demonstrates that SLA-MORL achieves 67.2% reduction in training time for deadline-critical jobs, 68.8% reduction in costs for budget-constrained workloads, and 73.4% improvement in overall SLA compliance compared to static baselines. By addressing both cold-start inefficiency and dynamic adaptation challenges, SLA-MORL provides a practical solution for cloud resource management that balances performance, cost, and reliability in modern ML training environments.