Architectural Adversarial Robustness: The Case for Deep Pursuit

TL;DR

This paper introduces a novel deep pursuit method that models all network layers as a single global optimization problem, enhancing adversarial robustness in deep neural networks, including residual architectures.

Contribution

It proposes a new deep pursuit approach that improves robustness by considering all layers simultaneously as a unified optimization problem.

Findings

Enhanced adversarial robustness demonstrated experimentally.

Applicable to deep architectures with skip connections.

Outperforms previous layer-wise pursuit methods.

Abstract

Despite their unmatched performance, deep neural networks remain susceptible to targeted attacks by nearly imperceptible levels of adversarial noise. While the underlying cause of this sensitivity is not well understood, theoretical analyses can be simplified by reframing each layer of a feed-forward network as an approximate solution to a sparse coding problem. Iterative solutions using basis pursuit are theoretically more stable and have improved adversarial robustness. However, cascading layer-wise pursuit implementations suffer from error accumulation in deeper networks. In contrast, our new method of deep pursuit approximates the activations of all layers as a single global optimization problem, allowing us to consider deeper, real-world architectures with skip connections such as residual networks. Experimentally, our approach demonstrates improved robustness to adversarial noise.