Gabor Layers Enhance Network Robustness

TL;DR

Replacing initial layers of deep networks with Gabor layers significantly improves robustness against adversarial attacks while maintaining high accuracy, and a new regularizer further enhances this robustness.

Contribution

This paper introduces the use of Gabor layers in deep networks to improve adversarial robustness and develops a Lipschitz-based regularizer for further gains.

Findings

Gabor layers increase robustness across multiple architectures.

The regularizer consistently improves robustness.

Models retain high test accuracy with Gabor layers.

Abstract

We revisit the benefits of merging classical vision concepts with deep learning models. In particular, we explore the effect on robustness against adversarial attacks of replacing the first layers of various deep architectures with Gabor layers, i.e. convolutional layers with filters that are based on learnable Gabor parameters. We observe that architectures enhanced with Gabor layers gain a consistent boost in robustness over regular models and preserve high generalizing test performance, even though these layers come at a negligible increase in the number of parameters. We then exploit the closed form expression of Gabor filters to derive an expression for a Lipschitz constant of such filters, and harness this theoretical result to develop a regularizer we use during training to further enhance network robustness. We conduct extensive experiments with various architectures (LeNet, AlexNet, VGG16 and WideResNet) on several datasets (MNIST, SVHN, CIFAR10 and CIFAR100) and demonstrate large empirical robustness gains. Furthermore, we experimentally show how our regularizer provides consistent robustness improvements.