Extreme Value Preserving Networks

TL;DR

This paper introduces EVPNets, a new CNN architecture inspired by SIFT's extreme value properties, which enhances robustness against adversarial attacks while maintaining accuracy.

Contribution

The paper proposes EVPNets, integrating scale-space extreme value concepts from SIFT into CNNs to improve adversarial robustness without sacrificing accuracy.

Findings

EVPNets achieve comparable or better accuracy than traditional CNNs.

EVPNets demonstrate significantly improved robustness against various adversarial attacks.

EVPNets do not require adversarial training to enhance robustness.

Abstract

Recent evidence shows that convolutional neural networks (CNNs) are biased towards textures so that CNNs are non-robust to adversarial perturbations over textures, while traditional robust visual features like SIFT (scale-invariant feature transforms) are designed to be robust across a substantial range of affine distortion, addition of noise, etc with the mimic of human perception nature. This paper aims to leverage good properties of SIFT to renovate CNN architectures towards better accuracy and robustness. We borrow the scale-space extreme value idea from SIFT, and propose extreme value preserving networks (EVPNets). Experiments demonstrate that EVPNets can achieve similar or better accuracy than conventional CNNs, while achieving much better robustness on a set of adversarial attacks (FGSM,PGD,etc) even without adversarial training.