Toward Accurate Platform-Aware Performance Modeling for Deep Neural Networks

TL;DR

This paper introduces PerfNetV2, a machine learning-based performance model that accurately predicts neural network inference and training times across various GPU accelerators, including unseen devices, aiding system optimization.

Contribution

PerfNetV2 improves accuracy over previous models by detailed host-accelerator interaction modeling and architecture enhancements, enabling reliable performance predictions on new hardware.

Findings

Achieves mean absolute percentage error within 13.1% on multiple CNNs.

Outperforms previous models with errors as high as 200%.

Capable of predicting performance on unseen GPU devices.

Abstract

In this paper, we provide a fine-grain machine learning-based method, PerfNetV2, which improves the accuracy of our previous work for modeling the neural network performance on a variety of GPU accelerators. Given an application, the proposed method can be used to predict the inference time and training time of the convolutional neural networks used in the application, which enables the system developer to optimize the performance by choosing the neural networks and/or incorporating the hardware accelerators to deliver satisfactory results in time. Furthermore, the proposed method is capable of predicting the performance of an unseen or non-existing device, e.g. a new GPU which has a higher operating frequency with less processor cores, but more memory capacity. This allows a system developer to quickly search the hardware design space and/or fine-tune the system configuration. Compared to the previous works, PerfNetV2 delivers more accurate results by modeling detailed host-accelerator interactions in executing the full neural networks and improving the architecture of the machine learning model used in the predictor. Our case studies show that PerfNetV2 yields a mean absolute percentage error within 13.1% on LeNet, AlexNet, and VGG16 on NVIDIA GTX-1080Ti, while the error rate on a previous work published in ICBD 2018 could be as large as 200%.