Layer-wise Regularized Adversarial Training using Layers Sustainability Analysis (LSA) framework

TL;DR

This paper introduces the Layer Sustainability Analysis (LSA) framework to identify vulnerable layers in neural networks and proposes a layer-wise regularization method (AT-LR) to enhance robustness against adversarial attacks, improving accuracy on benchmark datasets.

Contribution

The paper presents a novel LSA framework for analyzing layer vulnerability and introduces a layer-wise regularization approach (AT-LR) to improve adversarial robustness of neural networks.

Findings

AT-LR increases classification accuracy by up to 21.79% on MNIST.

LSA effectively identifies most vulnerable layers in neural networks.

Proposed methods perform well on multilayer perceptron and CNN architectures.

Abstract

Deep neural network models are used today in various applications of artificial intelligence, the strengthening of which, in the face of adversarial attacks is of particular importance. An appropriate solution to adversarial attacks is adversarial training, which reaches a trade-off between robustness and generalization. This paper introduces a novel framework (Layer Sustainability Analysis (LSA)) for the analysis of layer vulnerability in an arbitrary neural network in the scenario of adversarial attacks. LSA can be a helpful toolkit to assess deep neural networks and to extend the adversarial training approaches towards improving the sustainability of model layers via layer monitoring and analysis. The LSA framework identifies a list of Most Vulnerable Layers (MVL list) of the given network. The relative error, as a comparison measure, is used to evaluate representation sustainability of each layer against adversarial inputs. The proposed approach for obtaining robust neural networks to fend off adversarial attacks is based on a layer-wise regularization (LR) over LSA proposal(s) for adversarial training (AT); i.e. the AT-LR procedure. AT-LR could be used with any benchmark adversarial attack to reduce the vulnerability of network layers and to improve conventional adversarial training approaches. The proposed idea performs well theoretically and experimentally for state-of-the-art multilayer perceptron and convolutional neural network architectures. Compared with the AT-LR and its corresponding base adversarial training, the classification accuracy of more significant perturbations increased by 16.35%, 21.79%, and 10.730% on Moon, MNIST, and CIFAR-10 benchmark datasets, respectively. The LSA framework is available and published at https://github.com/khalooei/LSA.