Performance Prediction for Convolutional Neural Networks in Edge Devices

TL;DR

This paper evaluates five machine learning methods for predicting CNN execution time on edge GPUs, highlighting XGBoost's superior accuracy and efficiency in aiding CNN selection under hardware constraints.

Contribution

It compares and analyzes five ML-based prediction methods for CNN performance on edge devices, emphasizing XGBoost's accuracy and practical training considerations.

Findings

XGBoost achieves less than 14.73% average prediction error.

XGBoost requires less training effort compared to RF.

OLS, MLP, and SVR are less accurate for CNN performance estimation.

Abstract

Running Convolutional Neural Network (CNN) based applications on edge devices near the source of data can meet the latency and privacy challenges. However due to their reduced computing resources and their energy constraints, these edge devices can hardly satisfy CNN needs in processing and data storage. For these platforms, choosing the CNN with the best trade-off between accuracy and execution time while respecting Hardware constraints is crucial. In this paper, we present and compare five (5) of the widely used Machine Learning based methods for execution time prediction of CNNs on two (2) edge GPU platforms. For these 5 methods, we also explore the time needed for their training and tuning their corresponding hyperparameters. Finally, we compare times to run the prediction models on different platforms. The utilization of these methods will highly facilitate design space exploration by providing quickly the best CNN on a target edge GPU. Experimental results show that eXtreme Gradient Boosting (XGBoost) provides a less than 14.73% average prediction error even for unexplored and unseen CNN models' architectures. Random Forest (RF) depicts comparable accuracy but needs more effort and time to be trained. The other 3 approaches (OLS, MLP and SVR) are less accurate for CNN performances estimation.