Spectral Representations for Convolutional Neural Networks

TL;DR

This paper explores spectral domain representations for CNNs, introducing spectral pooling, stochastic regularization, and complex-coefficient parameterization, which improve information retention, training efficiency, and optimization.

Contribution

It presents novel spectral domain techniques for CNNs, including spectral pooling and regularization, enhancing model performance and training speed.

Findings

Spectral pooling preserves more information than traditional pooling.

Spectral regularization via randomized resolution improves generalization.

Complex-coefficient spectral parameterization accelerates CNN training.

Abstract

Discrete Fourier transforms provide a significant speedup in the computation of convolutions in deep learning. In this work, we demonstrate that, beyond its advantages for efficient computation, the spectral domain also provides a powerful representation in which to model and train convolutional neural networks (CNNs). We employ spectral representations to introduce a number of innovations to CNN design. First, we propose spectral pooling, which performs dimensionality reduction by truncating the representation in the frequency domain. This approach preserves considerably more information per parameter than other pooling strategies and enables flexibility in the choice of pooling output dimensionality. This representation also enables a new form of stochastic regularization by randomized modification of resolution. We show that these methods achieve competitive results on classification and approximation tasks, without using any dropout or max-pooling. Finally, we demonstrate the effectiveness of complex-coefficient spectral parameterization of convolutional filters. While this leaves the underlying model unchanged, it results in a representation that greatly facilitates optimization. We observe on a variety of popular CNN configurations that this leads to significantly faster convergence during training.