Deep Neural Networks Based Weight Approximation and Computation Reuse for 2-D Image Classification

TL;DR

This paper proposes a novel method combining approximate computing and data reuse in DNNs to enhance image classification efficiency on resource-constrained devices, significantly reducing parameters and hardware requirements with minimal accuracy loss.

Contribution

It introduces a weight approximation technique during training that enables computational and data reuse, leading to substantial reductions in parameters and hardware resources for DNN inference.

Findings

Achieved 1211.3x reduction in parameters on LeNet 5 with less than 0.9% accuracy loss.

Saved 54% of adders and multipliers compared to the Row Stationary method.

Reduced memory size and access requirements, suitable for IoT edge devices.

Abstract

Deep Neural Networks (DNNs) are computationally and memory intensive, which makes their hardware implementation a challenging task especially for resource constrained devices such as IoT nodes. To address this challenge, this paper introduces a new method to improve DNNs performance by fusing approximate computing with data reuse techniques to be used for image recognition applications. DNNs weights are approximated based on the linear and quadratic approximation methods during the training phase, then, all of the weights are replaced with the linear/quadratic coefficients to execute the inference in a way where different weights could be computed using the same coefficients. This leads to a repetition of the weights across the processing element (PE) array, which in turn enables the reuse of the DNN sub-computations (computational reuse) and leverage the same data (data reuse) to reduce DNNs computations, memory accesses, and improve energy efficiency albeit at the cost of increased training time. Complete analysis for both MNIST and CIFAR 10 datasets is presented for image recognition , where LeNet 5 revealed a reduction in the number of parameters by a factor of 1211.3x with a drop of less than 0.9% in accuracy. When compared to the state of the art Row Stationary (RS) method, the proposed architecture saved 54% of the total number of adders and multipliers needed. Overall, the proposed approach is suitable for IoT edge devices as it reduces the memory size requirement as well as the number of needed memory accesses.