ScaleDL: Towards Scalable and Efficient Runtime Prediction for Distributed Deep Learning Workloads

TL;DR

ScaleDL is a new framework that combines nonlinear layer-wise modeling with graph neural networks to accurately predict DNN runtimes across architectures while reducing data collection costs.

Contribution

It introduces a novel approach integrating GNNs and D-optimal data sampling to improve prediction accuracy and generalizability in distributed deep learning workloads.

Findings

Achieves 6x lower MRE than baselines

Achieves 5x lower RMSE than baselines

Demonstrates effectiveness across five DNN models

Abstract

Deep neural networks (DNNs) form the cornerstone of modern AI services, supporting a wide range of applications, including autonomous driving, chatbots, and recommendation systems. As models increase in size and complexity, DNN workloads such as training and inference tasks impose unprecedented demands on distributed computing resources, making accurate runtime prediction essential for optimizing development and resource allocation. Traditional methods rely on additive computational unit models, limiting their accuracy and generalizability. In contrast, graph-enhanced modeling improves performance but significantly increases data collection costs. Therefore, there is a critical need for a method that strikes a balance between accuracy, generalizability, and data collection costs. To address these challenges, we propose ScaleDL, a novel runtime prediction framework that combines nonlinear layer-wise modeling with graph neural network (GNN)-based cross-layer interaction mechanism, enabling accurate DNN runtime prediction and hierarchical generalizability across different network architectures. Additionally, we employ the D-optimal method to reduce data collection costs. Experiments on the workloads of five popular DNN models demonstrate that ScaleDL enhances runtime prediction accuracy and generalizability, achieving 6 times lower MRE and 5 times lower RMSE compared to baseline models.