On the Noise Stability and Robustness of Adversarially Trained Networks on NVM Crossbars

TL;DR

This paper investigates how combining adversarial training with the intrinsic robustness of NVM crossbar hardware affects the noise stability and robustness of DNNs, revealing trade-offs and potential gains in adversarial defense.

Contribution

It analyzes the noise stability of adversarially trained DNNs on NVM crossbars and demonstrates how hardware and training parameters influence robustness and performance.

Findings

Adversarially trained networks have lower SNR and are more sensitive to hardware noise.

Adversarial training combined with NVM hardware yields 20-30% robustness gains against black-box attacks.

Robustness gains against white-box PGD attacks are 5-10% when attack and training perturbations are mismatched.

Abstract

Applications based on Deep Neural Networks (DNNs) have grown exponentially in the past decade. To match their increasing computational needs, several Non-Volatile Memory (NVM) crossbar based accelerators have been proposed. Recently, researchers have shown that apart from improved energy efficiency and performance, such approximate hardware also possess intrinsic robustness for defense against adversarial attacks. Prior works quantified this intrinsic robustness for vanilla DNNs trained on unperturbed inputs. However, adversarial training of DNNs is the benchmark technique for robustness, and sole reliance on intrinsic robustness of the hardware may not be sufficient. In this work, we explore the design of robust DNNs through the amalgamation of adversarial training and intrinsic robustness of NVM crossbar-based analog hardware. First, we study the noise stability of such networks on unperturbed inputs and observe that internal activations of adversarially trained networks have lower Signal-to-Noise Ratio (SNR), and are sensitive to noise compared to vanilla networks. As a result, they suffer on average 2x performance degradation due to the approximate computations on analog hardware. Noise stability analyses show the instability of adversarially trained DNNs. On the other hand, for adversarial images generated using Square Black Box attacks, ResNet-10/20 adversarially trained on CIFAR-10/100 display a robustness gain of 20-30%. For adversarial images generated using Projected-Gradient-Descent (PGD) White-Box attacks, adversarially trained DNNs present a 5-10% gain in robust accuracy due to underlying NVM crossbar when $\epsilon_{attack}$ is greater than $\epsilon_{train}$. Our results indicate that implementing adversarially trained networks on analog hardware requires careful calibration between hardware non-idealities and $\epsilon_{train}$ for optimum robustness and performance.