Low-Energy Convolutional Neural Networks (CNNs) using Hadamard Method

TL;DR

This paper introduces a Hadamard transformation-based approach for CNNs that significantly reduces energy consumption in low-power devices, maintaining accuracy on simple datasets while highlighting limitations on more complex data.

Contribution

The paper proposes a novel Hadamard method as an energy-efficient alternative to traditional convolution operations in CNNs for IoT applications.

Findings

Energy savings demonstrated compared to convolution layers

Maintains accuracy on MNIST dataset

Reduced accuracy on CIFAR10 dataset due to complexity

Abstract

The growing demand for the internet of things (IoT) makes it necessary to implement computer vision tasks such as object recognition in low-power devices. Convolutional neural networks (CNNs) are a potential approach for object recognition and detection. However, the convolutional layer in CNN consumes significant energy compared to the fully connected layers. To mitigate this problem, a new approach based on the Hadamard transformation as an alternative to the convolution operation is demonstrated using two fundamental datasets, MNIST and CIFAR10. The mathematical expression of the Hadamard method shows the clear potential to save energy consumption compared to convolutional layers, which are helpful with BigData applications. In addition, to the test accuracy of the MNIST dataset, the Hadamard method performs similarly to the convolution method. In contrast, with the CIFAR10 dataset, test data accuracy is dropped (due to complex data and multiple channels) compared to the convolution method. Finally, the demonstrated method is helpful for other computer vision tasks when the kernel size is smaller than the input image size.