Local Linearity and Double Descent in Catastrophic Overfitting

TL;DR

This paper investigates the role of local linearity and orthogonality in preventing catastrophic overfitting during adversarial training, and uncovers the double descent phenomenon in this context.

Contribution

It demonstrates that high local linearity is sufficient but not necessary to prevent overfitting, introduces a regularization to enforce weight orthogonality, and identifies double descent in adversarial training.

Findings

High local linearity can prevent catastrophic overfitting.

Orthogonal weight regularization relates to local linearity.

Double descent occurs during adversarial training.

Abstract

Catastrophic overfitting is a phenomenon observed during Adversarial Training (AT) with the Fast Gradient Sign Method (FGSM) where the test robustness steeply declines over just one epoch in the training stage. Prior work has attributed this loss in robustness to a sharp decrease in $\textit{local linearity}$ of the neural network with respect to the input space, and has demonstrated that introducing a local linearity measure as a regularization term prevents catastrophic overfitting. Using a simple neural network architecture, we experimentally demonstrate that maintaining high local linearity might be $\textit{sufficient}$ to prevent catastrophic overfitting but is not $\textit{necessary.}$ Further, inspired by Parseval networks, we introduce a regularization term to AT with FGSM to make the weight matrices of the network orthogonal and study the connection between orthogonality of the network weights and local linearity. Lastly, we identify the $\textit{double descent}$ phenomenon during the adversarial training process.