Fix your downsampling ASAP! Be natively more robust via Aliasing and Spectral Artifact free Pooling

TL;DR

This paper introduces a frequency domain-based downsampling method called ASAP that eliminates aliasing and spectral artifacts in CNNs, leading to more robust models without sacrificing accuracy.

Contribution

It proposes a novel alias-free downsampling technique, ASAP, that improves CNN robustness by addressing spectral artifacts in the frequency domain.

Findings

Enhanced robustness against corruptions and adversarial attacks

Maintains comparable accuracy to baseline models

Effective across multiple CNN architectures and datasets

Abstract

Convolutional Neural Networks (CNNs) are successful in various computer vision tasks. From an image and signal processing point of view, this success is counter-intuitive, as the inherent spatial pyramid design of most CNNs is apparently violating basic signal processing laws, i.e. the Sampling Theorem in their downsampling operations. This issue has been broadly neglected until recent work in the context of adversarial attacks and distribution shifts showed that there is a strong correlation between the vulnerability of CNNs and aliasing artifacts induced by bandlimit-violating downsampling. As a remedy, we propose an alias-free downsampling operation in the frequency domain, denoted Frequency Low Cut Pooling (FLC Pooling) which we further extend to Aliasing and Sinc Artifact-free Pooling (ASAP). ASAP is alias-free and removes further artifacts from sinc-interpolation. Our experimental evaluation on ImageNet-1k, ImageNet-C and CIFAR datasets on various CNN architectures demonstrates that networks using FLC Pooling and ASAP as downsampling methods learn more stable features as measured by their robustness against common corruptions and adversarial attacks, while maintaining a clean accuracy similar to the respective baseline models.