Robust Classification by Coupling Data Mollification with Label Smoothing

TL;DR

This paper proposes a simple yet effective training augmentation that combines data mollification with label smoothing to improve neural network robustness and uncertainty estimation against corrupted images.

Contribution

It introduces a novel coupling of data mollification with label smoothing, inspired by diffusion models, to enhance model robustness and uncertainty quantification.

Findings

Improved robustness on CIFAR, TinyImageNet, and ImageNet benchmarks.

Enhanced uncertainty quantification in corrupted image scenarios.

Method is easy to implement with negligible overheads.

Abstract

Introducing training-time augmentations is a key technique to enhance generalization and prepare deep neural networks against test-time corruptions. Inspired by the success of generative diffusion models, we propose a novel approach of coupling data mollification, in the form of image noising and blurring, with label smoothing to align predicted label confidences with image degradation. The method is simple to implement, introduces negligible overheads, and can be combined with existing augmentations. We demonstrate improved robustness and uncertainty quantification on the corrupted image benchmarks of CIFAR, TinyImageNet and ImageNet datasets.