Benchmark Analysis of Representative Deep Neural Network Architectures

TL;DR

This paper provides a comprehensive benchmark analysis of popular deep neural network architectures for image recognition, evaluating their performance across multiple indices on different hardware platforms to guide research and practical deployment.

Contribution

It offers a detailed comparison of DNN architectures across various performance metrics and hardware setups, aiding informed selection for different application needs.

Findings

Different DNNs vary significantly in accuracy, complexity, and inference time.

Hardware impacts DNN performance and resource usage.

The study provides a resource for future research and practical deployment decisions.

Abstract

This work presents an in-depth analysis of the majority of the deep neural networks (DNNs) proposed in the state of the art for image recognition. For each DNN multiple performance indices are observed, such as recognition accuracy, model complexity, computational complexity, memory usage, and inference time. The behavior of such performance indices and some combinations of them are analyzed and discussed. To measure the indices we experiment the use of DNNs on two different computer architectures, a workstation equipped with a NVIDIA Titan X Pascal and an embedded system based on a NVIDIA Jetson TX1 board. This experimentation allows a direct comparison between DNNs running on machines with very different computational capacity. This study is useful for researchers to have a complete view of what solutions have been explored so far and in which research directions are worth exploring in the future; and for practitioners to select the DNN architecture(s) that better fit the resource constraints of practical deployments and applications. To complete this work, all the DNNs, as well as the software used for the analysis, are available online.