A Theoretical Perspective on Subnetwork Contributions to Adversarial Robustness

TL;DR

This paper introduces a theoretical framework to analyze how the robustness of a subnetwork influences the entire neural network's resistance to adversarial attacks, supported by empirical validation across various architectures and datasets.

Contribution

It develops the concept of semirobustness and provides a theoretical analysis linking subnetwork robustness to overall network robustness, enhancing understanding of adversarial defenses.

Findings

Robust subnetworks can promote full-network robustness.

Layer dependencies are crucial for robustness transfer.

Empirical validation across multiple architectures and attacks.

Abstract

The robustness of deep neural networks (DNNs) against adversarial attacks has been studied extensively in hopes of both better understanding how deep learning models converge and in order to ensure the security of these models in safety-critical applications. Adversarial training is one approach to strengthening DNNs against adversarial attacks, and has been shown to offer a means for doing so at the cost of applying computationally expensive training methods to the entire model. To better understand these attacks and facilitate more efficient adversarial training, in this paper we develop a novel theoretical framework that investigates how the adversarial robustness of a subnetwork contributes to the robustness of the entire network. To do so we first introduce the concept of semirobustness, which is a measure of the adversarial robustness of a subnetwork. Building on this concept, we then provide a theoretical analysis to show that if a subnetwork is semirobust and there is a sufficient dependency between it and each subsequent layer in the network, then the remaining layers are also guaranteed to be robust. We validate these findings empirically across multiple DNN architectures, datasets, and adversarial attacks. Experiments show the ability of a robust subnetwork to promote full-network robustness, and investigate the layer-wise dependencies required for this full-network robustness to be achieved.