NoisyMix: Boosting Model Robustness to Common Corruptions

TL;DR

NoisyMix is a new training scheme that enhances neural network robustness and calibration by using noisy augmentations in input and feature spaces, outperforming existing methods on various benchmarks.

Contribution

Introduces NoisyMix, a novel data augmentation-based training method that improves robustness and calibration of neural networks, supported by theoretical analysis.

Findings

Models trained with NoisyMix are more robust to corruptions.

NoisyMix improves in-domain accuracy and calibration.

Demonstrated effectiveness on ImageNet benchmarks.

Abstract

For many real-world applications, obtaining stable and robust statistical performance is more important than simply achieving state-of-the-art predictive test accuracy, and thus robustness of neural networks is an increasingly important topic. Relatedly, data augmentation schemes have been shown to improve robustness with respect to input perturbations and domain shifts. Motivated by this, we introduce NoisyMix, a novel training scheme that promotes stability as well as leverages noisy augmentations in input and feature space to improve both model robustness and in-domain accuracy. NoisyMix produces models that are consistently more robust and that provide well-calibrated estimates of class membership probabilities. We demonstrate the benefits of NoisyMix on a range of benchmark datasets, including ImageNet-C, ImageNet-R, and ImageNet-P. Moreover, we provide theory to understand implicit regularization and robustness of NoisyMix.