Learning Convolutional Neural Networks in the Frequency Domain

TL;DR

This paper introduces CEMNet, a CNN trained in the frequency domain using element-wise multiplication to reduce computation, along with mechanisms to prevent overfitting and adapt common layers for frequency domain operation, demonstrating good performance on MNIST and CIFAR-10.

Contribution

The paper presents a novel frequency domain training method for CNNs, including a two-branches structure and adaptations of normalization and activation functions, reducing computational complexity.

Findings

CEMNet achieves competitive accuracy on MNIST and CIFAR-10.

Frequency domain training reduces convolution computation complexity.

The proposed mechanisms effectively prevent overfitting.

Abstract

Convolutional neural network (CNN) has achieved impressive success in computer vision during the past few decades. The image convolution operation helps CNNs to get good performance on image-related tasks. However, the image convolution has high computation complexity and hard to be implemented. This paper proposes the CEMNet, which can be trained in the frequency domain. The most important motivation of this research is that we can use the straightforward element-wise multiplication operation to replace the image convolution in the frequency domain based on the Cross-Correlation Theorem, which obviously reduces the computation complexity. We further introduce a Weight Fixation mechanism to alleviate the problem of over-fitting, and analyze the working behavior of Batch Normalization, Leaky ReLU, and Dropout in the frequency domain to design their counterparts for CEMNet. Also, to deal with complex inputs brought by Discrete Fourier Transform, we design a two-branches network structure for CEMNet. Experimental results imply that CEMNet achieves good performance on MNIST and CIFAR-10 databases.