Theoretical Understanding of Learning from Adversarial Perturbations

TL;DR

This paper develops a theoretical framework to understand how adversarial perturbations contain class features and influence neural network generalization, revealing that even minimal perturbations can encode sufficient class information.

Contribution

It introduces a theoretical model demonstrating that adversarial perturbations include class features and affect decision boundaries similarly to standard samples.

Findings

Perturbations of a few pixels contain enough class features for generalization.

The decision boundary from learning with perturbations closely matches that from standard samples.

Theoretical insights explain transferability and deception of adversarial examples.

Abstract

It is not fully understood why adversarial examples can deceive neural networks and transfer between different networks. To elucidate this, several studies have hypothesized that adversarial perturbations, while appearing as noises, contain class features. This is supported by empirical evidence showing that networks trained on mislabeled adversarial examples can still generalize well to correctly labeled test samples. However, a theoretical understanding of how perturbations include class features and contribute to generalization is limited. In this study, we provide a theoretical framework for understanding learning from perturbations using a one-hidden-layer network trained on mutually orthogonal samples. Our results highlight that various adversarial perturbations, even perturbations of a few pixels, contain sufficient class features for generalization. Moreover, we reveal that the decision boundary when learning from perturbations matches that from standard samples except for specific regions under mild conditions. The code is available at https://github.com/s-kumano/learning-from-adversarial-perturbations.