PROFET: Profiling-based CNN Training Latency Prophet for GPU Cloud Instances

TL;DR

PROFET is a system that accurately predicts CNN training latency on various GPUs without needing detailed architecture info, helping cloud providers optimize training environments efficiently.

Contribution

It introduces a latency prediction method that does not depend on CNN architecture details, suitable for public cloud deployment and adaptable to evolving cloud services.

Findings

PROFET achieves higher prediction accuracy than existing methods.

The system demonstrates practical utility in cloud training environments.

Evaluation confirms robustness across different GPU models.

Abstract

Training a Convolutional Neural Network (CNN) model typically requires significant computing power, and cloud computing resources are widely used as a training environment. However, it is difficult for CNN algorithm developers to keep up with system updates and apply them to their training environment due to quickly evolving cloud services. Thus, it is important for cloud computing service vendors to design and deliver an optimal training environment for various training tasks to lessen system operation management overhead of algorithm developers. To achieve the goal, we propose PROFET, which can predict the training latency of arbitrary CNN implementation on various Graphical Processing Unit (GPU) devices to develop a cost-effective and time-efficient training cloud environment. Different from the previous training latency prediction work, PROFET does not rely on the implementation details of the CNN architecture, and it is suitable for use in a public cloud environment. Thorough evaluations reveal the superior prediction accuracy of PROFET compared to the state-of-the-art related work, and the demonstration service presents the practicality of the proposed system.