Early Transferability of Adversarial Examples in Deep Neural Networks

TL;DR

This paper introduces and analyzes 'Early Transferability', a phenomenon where adversarial perturbations become aligned across different neural networks during the initial training stages, even before models improve in accuracy.

Contribution

It uncovers the early transferability phenomenon of adversarial examples in neural networks and provides experimental evidence and plausible explanations for this behavior.

Findings

Adversarial directions align early in training

Transferability occurs before accuracy improves

Alignment happens within first few training steps

Abstract

This paper will describe and analyze a new phenomenon that was not known before, which we call "Early Transferability". Its essence is that the adversarial perturbations transfer among different networks even at extremely early stages in their training. In fact, one can initialize two networks with two different independent choices of random weights and measure the angle between their adversarial perturbations after each step of the training. What we discovered was that these two adversarial directions started to align with each other already after the first few training steps (which typically use only a small fraction of the available training data), even though the accuracy of the two networks hadn't started to improve from their initial bad values due to the early stage of the training. The purpose of this paper is to present this phenomenon experimentally and propose plausible explanations for some of its properties.