Salient Information Preserving Adversarial Training Improves Clean and Robust Accuracy

TL;DR

SIP-AT is a novel adversarial training method that preserves salient image regions to improve both clean and robust accuracy, reducing the traditional trade-off in adversarial robustness.

Contribution

This work introduces SIP-AT, a salience-guided adversarial training technique that maintains meaningful features and enhances model performance on clean data without sacrificing robustness.

Findings

SIP-AT boosts clean accuracy across multiple datasets and architectures.

Models trained with SIP-AT maintain high robustness at various attack epsilon levels.

Human studies show increased difficulty in identifying perturbed images at low epsilon levels.

Abstract

In this work we introduce Salient Information Preserving Adversarial Training (SIP-AT), an intuitive method for relieving the robustness-accuracy trade-off incurred by traditional adversarial training. SIP-AT uses salient image regions to guide the adversarial training process in such a way that fragile features deemed meaningful by an annotator remain unperturbed during training, allowing models to learn highly predictive non-robust features without sacrificing overall robustness. This technique is compatible with both human-based and automatically generated salience estimates, allowing SIP-AT to be used as a part of human-driven model development without forcing SIP-AT to be reliant upon additional human data. We perform experiments across multiple datasets and architectures and demonstrate that SIP-AT is able to boost the clean accuracy of models while maintaining a high degree of robustness against attacks at multiple epsilon levels. We complement our central experiments with an observational study measuring the rate at which human subjects successfully identify perturbed images. This study helps build a more intuitive understanding of adversarial attack strength and demonstrates the heightened importance of low-epsilon robustness. Our results demonstrate the efficacy of SIP-AT and provide valuable insight into the risks posed by adversarial samples of various strengths.