Truly Scale-Equivariant Deep Nets with Fourier Layers

TL;DR

This paper introduces a novel Fourier layer-based architecture that achieves truly scale-equivariant deep neural networks with zero equivariance-error, addressing limitations of previous methods that lacked anti-aliasing considerations.

Contribution

The paper proposes a new architecture using Fourier layers to ensure true scale-equivariance by incorporating anti-aliasing in the discrete domain, achieving zero equivariance-error.

Findings

Achieves competitive classification accuracy on MNIST-scale and STL-10 datasets.

Maintains zero scale-equivariance-error in experiments.

Addresses anti-aliasing in scale transformations for CNNs.

Abstract

In computer vision, models must be able to adapt to changes in image resolution to effectively carry out tasks such as image segmentation; This is known as scale-equivariance. Recent works have made progress in developing scale-equivariant convolutional neural networks, e.g., through weight-sharing and kernel resizing. However, these networks are not truly scale-equivariant in practice. Specifically, they do not consider anti-aliasing as they formulate the down-scaling operation in the continuous domain. To address this shortcoming, we directly formulate down-scaling in the discrete domain with consideration of anti-aliasing. We then propose a novel architecture based on Fourier layers to achieve truly scale-equivariant deep nets, i.e., absolute zero equivariance-error. Following prior works, we test this model on MNIST-scale and STL-10 datasets. Our proposed model achieves competitive classification performance while maintaining zero equivariance-error.