NPAT Null-Space Projected Adversarial Training Towards Zero Deterioration

TL;DR

This paper introduces NPAT, a novel adversarial training method using null-space projection to enhance robustness against attacks with minimal impact on normal accuracy.

Contribution

It pioneers the use of null-space projection in adversarial training, proposing two algorithms (NPDA and NPGD) to improve robustness while preserving generalization.

Findings

Achieves comparable robustness to existing methods

Maintains high generalization performance

Effective on CIFAR10 and SVHN datasets

Abstract

To mitigate the susceptibility of neural networks to adversarial attacks, adversarial training has emerged as a prevalent and effective defense strategy. Intrinsically, this countermeasure incurs a trade-off, as it sacrifices the model's accuracy in processing normal samples. To reconcile the trade-off, we pioneer the incorporation of null-space projection into adversarial training and propose two innovative Null-space Projection based Adversarial Training(NPAT) algorithms tackling sample generation and gradient optimization, named Null-space Projected Data Augmentation (NPDA) and Null-space Projected Gradient Descent (NPGD), to search for an overarching optimal solutions, which enhance robustness with almost zero deterioration in generalization performance. Adversarial samples and perturbations are constrained within the null-space of the decision boundary utilizing a closed-form null-space projector, effectively mitigating threat of attack stemming from unreliable features. Subsequently, we conducted experiments on the CIFAR10 and SVHN datasets and reveal that our methodology can seamlessly combine with adversarial training methods and obtain comparable robustness while keeping generalization close to a high-accuracy model.