An Analysis of Deep Neural Network Models for Practical Applications

TL;DR

This paper provides a comprehensive analysis of deep neural network models focusing on practical metrics like accuracy, resource use, and energy consumption, revealing key relationships and constraints for efficient model design.

Contribution

It offers a detailed evaluation of DNNs considering both accuracy and resource metrics, highlighting the importance of energy constraints and operational efficiency in practical applications.

Findings

Power consumption is independent of batch size and architecture.

Accuracy and inference time have a hyperbolic relationship.

Energy constraints limit maximum accuracy and complexity.

Abstract

Since the emergence of Deep Neural Networks (DNNs) as a prominent technique in the field of computer vision, the ImageNet classification challenge has played a major role in advancing the state-of-the-art. While accuracy figures have steadily increased, the resource utilisation of winning models has not been properly taken into account. In this work, we present a comprehensive analysis of important metrics in practical applications: accuracy, memory footprint, parameters, operations count, inference time and power consumption. Key findings are: (1) power consumption is independent of batch size and architecture; (2) accuracy and inference time are in a hyperbolic relationship; (3) energy constraint is an upper bound on the maximum achievable accuracy and model complexity; (4) the number of operations is a reliable estimate of the inference time. We believe our analysis provides a compelling set of information that helps design and engineer efficient DNNs.