Robust Design of Deep Neural Networks against Adversarial Attacks based on Lyapunov Theory

TL;DR

This paper introduces a control theoretic approach using Lyapunov stability to enhance the robustness of deep neural networks against adversarial attacks, providing theoretical guarantees and empirical validation.

Contribution

It applies Lyapunov theory to prove local and global stability of DNNs, offering new bounds and insights into spectral norm regularization for robustness.

Findings

Lyapunov-based bounds on layer responses to adversarial perturbations

Spectral norm regularization's role in mitigating adversarial effects

Empirical validation across datasets and architectures

Abstract

Deep neural networks (DNNs) are vulnerable to subtle adversarial perturbations applied to the input. These adversarial perturbations, though imperceptible, can easily mislead the DNN. In this work, we take a control theoretic approach to the problem of robustness in DNNs. We treat each individual layer of the DNN as a nonlinear dynamical system and use Lyapunov theory to prove stability and robustness locally. We then proceed to prove stability and robustness globally for the entire DNN. We develop empirically tight bounds on the response of the output layer, or any hidden layer, to adversarial perturbations added to the input, or the input of hidden layers. Recent works have proposed spectral norm regularization as a solution for improving robustness against l2 adversarial attacks. Our results give new insights into how spectral norm regularization can mitigate the adversarial effects. Finally, we evaluate the power of our approach on a variety of data sets and network architectures and against some of the well-known adversarial attacks.