Enhancing Adversarial Training with Feature Separability

TL;DR

This paper introduces a novel adversarial training method called ATFS that enhances feature separability, leading to improved robustness and generalization of deep neural networks against adversarial attacks.

Contribution

The paper proposes ATFS, a new framework that improves feature representations by increasing intra-class similarity and inter-class variance during adversarial training.

Findings

ATFS significantly improves robustness against adversarial attacks.

Enhanced feature separability leads to better generalization on clean data.

Experimental results outperform existing adversarial training methods.

Abstract

Deep Neural Network (DNN) are vulnerable to adversarial attacks. As a countermeasure, adversarial training aims to achieve robustness based on the min-max optimization problem and it has shown to be one of the most effective defense strategies. However, in this work, we found that compared with natural training, adversarial training fails to learn better feature representations for either clean or adversarial samples, which can be one reason why adversarial training tends to have severe overfitting issues and less satisfied generalize performance. Specifically, we observe two major shortcomings of the features learned by existing adversarial training methods:(1) low intra-class feature similarity; and (2) conservative inter-classes feature variance. To overcome these shortcomings, we introduce a new concept of adversarial training graph (ATG) with which the proposed adversarial training with feature separability (ATFS) enables to coherently boost the intra-class feature similarity and increase inter-class feature variance. Through comprehensive experiments, we demonstrate that the proposed ATFS framework significantly improves both clean and robust performance.