Modeling Data Reuse in Deep Neural Networks by Taking Data-Types into Cognizance

TL;DR

This paper introduces a data type aware model to better estimate data reuse and energy efficiency in deep neural networks, addressing limitations of traditional arithmetic intensity metrics.

Contribution

It proposes a novel data type aware weighted arithmetic intensity model that improves data reuse estimation in DNNs compared to conventional methods.

Findings

Accurately models data reuse for various DNN architectures.

Better predicts energy efficiency of DNNs on GPU hardware.

Demonstrates generality using the central limit theorem.

Abstract

In recent years, researchers have focused on reducing the model size and number of computations (measured as "multiply-accumulate" or MAC operations) of DNNs. The energy consumption of a DNN depends on both the number of MAC operations and the energy efficiency of each MAC operation. The former can be estimated at design time; however, the latter depends on the intricate data reuse patterns and underlying hardware architecture. Hence, estimating it at design time is challenging. This work shows that the conventional approach to estimate the data reuse, viz. arithmetic intensity, does not always correctly estimate the degree of data reuse in DNNs since it gives equal importance to all the data types. We propose a novel model, termed "data type aware weighted arithmetic intensity" ($DI$), which accounts for the unequal importance of different data types in DNNs. We evaluate our model on 25 state-of-the-art DNNs on two GPUs. We show that our model accurately models data-reuse for all possible data reuse patterns for different types of convolution and different types of layers. We show that our model is a better indicator of the energy efficiency of DNNs. We also show its generality using the central limit theorem.