Performance Analysis of DNN Inference/Training with Convolution and non-Convolution Operations

TL;DR

This paper introduces SimDIT, a comprehensive performance analysis framework for ASIC-based deep learning accelerators that models both convolutional and non-convolutional operations during CNN inference and training, providing detailed performance insights.

Contribution

SimDIT is the first framework to accurately model both convolution and non-convolution operations for training and inference on ASIC accelerators, integrating with silicon flow for detailed performance metrics.

Findings

Non-convolution operations account for 59.5% of runtime in ResNet-50 training.

Optimized resource allocation yields 18X performance improvement in inference.

SimDIT provides detailed end-to-end performance statistics for CNN workloads.

Abstract

Today's performance analysis frameworks for deep learning accelerators suffer from two significant limitations. First, although modern convolutional neural network (CNNs) consist of many types of layers other than convolution, especially during training, these frameworks largely focus on convolution layers only. Second, these frameworks are generally targeted towards inference, and lack support for training operations. This work proposes a novel performance analysis framework, SimDIT, for general ASIC-based systolic hardware accelerator platforms. The modeling effort of SimDIT comprehensively covers convolution and non-convolution operations of both CNN inference and training on a highly parameterizable hardware substrate. SimDIT is integrated with a backend silicon implementation flow and provides detailed end-to-end performance statistics (i.e., data access cost, cycle counts, energy, and power) for executing CNN inference and training workloads. SimDIT-enabled performance analysis reveals that on a 64X64 processing array, non-convolution operations constitute 59.5% of total runtime for ResNet-50 training workload. In addition, by optimally distributing available off-chip DRAM bandwidth and on-chip SRAM resources, SimDIT achieves 18X performance improvement over a generic static resource allocation for ResNet-50 inference.