Pick the Right Edge Device: Towards Power and Performance Estimation of CUDA-based CNNs on GPGPUs

TL;DR

This paper introduces a machine learning approach to estimate power and performance of CUDA-based CNNs on GPGPUs, aiding engineers in selecting the most efficient device early in development.

Contribution

It presents a novel ML-based method for early power and performance estimation of CNNs on GPGPUs, improving device selection decisions.

Findings

Accurately predicts power consumption of CNNs on GPGPUs

Provides performance estimates to optimize device choice

Enables early-stage decision making for ML deployment

Abstract

The emergence of Machine Learning (ML) as a powerful technique has been helping nearly all fields of business to increase operational efficiency or to develop new value propositions. Besides the challenges of deploying and maintaining ML models, picking the right edge device (e.g., GPGPUs) to run these models (e.g., CNN with the massive computational process) is one of the most pressing challenges faced by organizations today. As the cost of renting (on Cloud) or purchasing an edge device is directly connected to the cost of final products or services, choosing the most efficient device is essential. However, this decision making requires deep knowledge about performance and power consumption of the ML models running on edge devices that must be identified at the early stage of ML workflow. In this paper, we present a novel ML-based approach that provides ML engineers with the early estimation of both power consumption and performance of CUDA-based CNNs on GPGPUs. The proposed approach empowers ML engineers to pick the most efficient GPGPU for a given CNN model at the early stage of development.