Frequency Pooling: Shift-Equivalent and Anti-Aliasing Downsampling

TL;DR

This paper introduces frequency pooling, a novel shift-equivalent and anti-aliasing pooling method for CNNs that enhances accuracy and robustness by transforming features into the frequency domain, removing high-frequency components, and transforming back.

Contribution

The paper proposes frequency pooling, the first pooling method that is both shift-equivalent and anti-aliasing, addressing limitations of traditional pooling techniques.

Findings

Frequency pooling improves CNN accuracy on image classification tasks.

Frequency pooling enhances robustness to input shifts.

Theoretical proof of shift-equivalence and anti-aliasing properties.

Abstract

Convolution utilizes a shift-equivalent prior of images, thus leading to great success in image processing tasks. However, commonly used poolings in convolutional neural networks (CNNs), such as max-pooling, average-pooling, and strided-convolution, are not shift-equivalent. Thus, the shift-equivalence of CNNs is destroyed when convolutions and poolings are stacked. Moreover, anti-aliasing is another essential property of poolings from the perspective of signal processing. However, recent poolings are neither shift-equivalent nor anti-aliasing. To address this issue, we propose a new pooling method that is shift-equivalent and anti-aliasing, named frequency pooling. Frequency pooling first transforms the features into the frequency domain, and then removes the frequency components beyond the Nyquist frequency. Finally, it transforms the features back to the spatial domain. We prove that frequency pooling is shift-equivalent and anti-aliasing based on the property of Fourier transform and Nyquist frequency. Experiments on image classification show that frequency pooling improves accuracy and robustness with respect to the shifts of CNNs.