On the Equivalence of Convolutional and Hadamard Networks using DFT

TL;DR

This paper demonstrates that convolutional networks can be transformed into Hadamard networks in the frequency domain using DFT, providing new insights into their computational equivalence and performance.

Contribution

The paper introduces frequency domain activation functions that convert convolutional networks into Hadamard networks, revealing their fundamental equivalence.

Findings

Convolutional and Hadamard networks are equivalent in the frequency domain.

Implementation details for frequency domain networks are provided.

Experimental results support the theoretical equivalence.

Abstract

In this paper we introduce activation functions that move the entire computation of Convolutional Networks into the frequency domain, where they are actually Hadamard Networks. To achieve this result we employ the properties of Discrete Fourier Transform. We present some implementation details and experimental results, as well as some insights into why convolutional networks perform well in learning use cases.