FrequencyLowCut Pooling -- Plug & Play against Catastrophic Overfitting
Julia Grabinski, Steffen Jung, Janis Keuper, Margret Keuper

TL;DR
This paper introduces FrequencyLowCut pooling, an aliasing-free down-sampling method that enhances CNN robustness against adversarial attacks and prevents catastrophic overfitting when integrated into existing architectures.
Contribution
The paper proposes a novel plug-and-play pooling operation that reduces aliasing, improving CNN robustness and training stability without additional hyper-parameters.
Findings
Significantly improves model robustness against adversarial attacks.
Prevents catastrophic overfitting during adversarial training.
Easily integrates into existing CNN architectures.
Abstract
Over the last years, Convolutional Neural Networks (CNNs) have been the dominating neural architecture in a wide range of computer vision tasks. From an image and signal processing point of view, this success might be a bit surprising as the inherent spatial pyramid design of most CNNs is apparently violating basic signal processing laws, i.e. Sampling Theorem in their down-sampling operations. However, since poor sampling appeared not to affect model accuracy, this issue has been broadly neglected until model robustness started to receive more attention. Recent work [17] in the context of adversarial attacks and distribution shifts, showed after all, that there is a strong correlation between the vulnerability of CNNs and aliasing artifacts induced by poor down-sampling operations. This paper builds on these findings and introduces an aliasing free down-sampling operation which can…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdversarial Robustness in Machine Learning · Anomaly Detection Techniques and Applications · Integrated Circuits and Semiconductor Failure Analysis
