Label Augmentation for Neural Networks Robustness

TL;DR

This paper introduces Label Augmentation (LA), a novel method that significantly improves neural network robustness against both natural and adversarial out-of-distribution perturbations, while also enhancing uncertainty estimation.

Contribution

The study proposes Label Augmentation (LA), a new approach that enhances robustness to common and adversarial perturbations and improves uncertainty estimation in neural networks.

Findings

Clean error rate improved by up to 23.29%.

Robustness under common corruptions increased by up to 24.23%.

Adversarial robustness improved by up to 53.18% for FGSM and 24.46% for PGD attacks.

Abstract

Out-of-distribution generalization can be categorized into two types: common perturbations arising from natural variations in the real world and adversarial perturbations that are intentionally crafted to deceive neural networks. While deep neural networks excel in accuracy under the assumption of identical distributions between training and test data, they often encounter out-of-distribution scenarios resulting in a significant decline in accuracy. Data augmentation methods can effectively enhance robustness against common corruptions, but they typically fall short in improving robustness against adversarial perturbations. In this study, we develop Label Augmentation (LA), which enhances robustness against both common and intentional perturbations and improves uncertainty estimation. Our findings indicate a Clean error rate improvement of up to 23.29% when employing LA in comparisons to the baseline. Additionally, it enhances robustness under common corruptions benchmark by up to 24.23%. When tested against FGSM and PGD attacks, improvements in adversarial robustness are noticeable, with enhancements of up to 53.18% for FGSM and 24.46% for PGD attacks.