Efficient Transformations in Deep Learning Convolutional Neural Networks

TL;DR

This paper explores integrating signal processing transforms like WHT into ResNet50 to improve energy efficiency and accuracy in image classification, demonstrating significant energy savings and accuracy gains on CIFAR-100.

Contribution

It introduces a novel method of embedding WHT into CNN layers, showing improved energy efficiency and accuracy over standard models.

Findings

WHT reduces energy consumption by over 99%.

Incorporating WHT improves classification accuracy from 66% to 79%.

Transform integration offers a promising approach for energy-constrained applications.

Abstract

This study investigates the integration of signal processing transformations -- Fast Fourier Transform (FFT), Walsh-Hadamard Transform (WHT), and Discrete Cosine Transform (DCT) -- within the ResNet50 convolutional neural network (CNN) model for image classification. The primary objective is to assess the trade-offs between computational efficiency, energy consumption, and classification accuracy during training and inference. Using the CIFAR-100 dataset (100 classes, 60,000 images), experiments demonstrated that incorporating WHT significantly reduced energy consumption while improving accuracy. Specifically, a baseline ResNet50 model achieved a testing accuracy of 66%, consuming an average of 25,606 kJ per model. In contrast, a modified ResNet50 incorporating WHT in the early convolutional layers achieved 74% accuracy, and an enhanced version with WHT applied to both early and late layers achieved 79% accuracy, with an average energy consumption of only 39 kJ per model. These results demonstrate the potential of WHT as a highly efficient and effective approach for energy-constrained CNN applications.