On the Intrinsic Robustness of NVM Crossbars Against Adversarial Attacks

TL;DR

This paper investigates how the inherent non-idealities of NVM crossbar hardware provide intrinsic robustness against adversarial attacks in deep learning, with significant improvements in adversarial accuracy observed in non-adaptive scenarios.

Contribution

The study reveals that analog NVM crossbars inherently reduce the effectiveness of adversarial attacks, offering a new perspective on hardware-based robustness in neural network accelerators.

Findings

Non-idealities in NVM crossbars decrease attack success rates.

Intrinsic robustness varies across datasets, with up to 35% accuracy improvement.

Adaptive attacks can overcome hardware-induced robustness.

Abstract

The increasing computational demand of Deep Learning has propelled research in special-purpose inference accelerators based on emerging non-volatile memory (NVM) technologies. Such NVM crossbars promise fast and energy-efficient in-situ Matrix Vector Multiplication (MVM) thus alleviating the long-standing von Neuman bottleneck in today's digital hardware. However, the analog nature of computing in these crossbars is inherently approximate and results in deviations from ideal output values, which reduces the overall performance of Deep Neural Networks (DNNs) under normal circumstances. In this paper, we study the impact of these non-idealities under adversarial circumstances. We show that the non-ideal behavior of analog computing lowers the effectiveness of adversarial attacks, in both Black-Box and White-Box attack scenarios. In a non-adaptive attack, where the attacker is unaware of the analog hardware, we observe that analog computing offers a varying degree of intrinsic robustness, with a peak adversarial accuracy improvement of 35.34%, 22.69%, and 9.90% for white box PGD (epsilon=1/255, iter=30) for CIFAR-10, CIFAR-100, and ImageNet respectively. We also demonstrate "Hardware-in-Loop" adaptive attacks that circumvent this robustness by utilizing the knowledge of the NVM model.