One Neuron to Fool Them All

TL;DR

This paper introduces a neuron-level perspective on adversarial robustness, revealing that single neurons can be highly influential in model vulnerability and proposing a regularization method to improve robustness.

Contribution

It proposes a new neuron sensitivity measure for understanding adversarial susceptibility and introduces a regularization technique to enhance model robustness across perturbation types.

Findings

Single neurons can be as effective as the full model in adversarial attacks.

Sensitive neurons exhibit distinctive properties that can be exploited for robustness.

Regularization based on neuron sensitivity improves model robustness without sacrificing accuracy.

Abstract

Despite vast research in adversarial examples, the root causes of model susceptibility are not well understood. Instead of looking at attack-specific robustness, we propose a notion that evaluates the sensitivity of individual neurons in terms of how robust the model's output is to direct perturbations of that neuron's output. Analyzing models from this perspective reveals distinctive characteristics of standard as well as adversarially-trained robust models, and leads to several curious results. In our experiments on CIFAR-10 and ImageNet, we find that attacks using a loss function that targets just a single sensitive neuron find adversarial examples nearly as effectively as ones that target the full model. We analyze the properties of these sensitive neurons to propose a regularization term that can help a model achieve robustness to a variety of different perturbation constraints while maintaining accuracy on natural data distributions. Code for all our experiments is available at https://github.com/iamgroot42/sauron .