Soft Augmentation for Image Classification

TL;DR

This paper introduces soft augmentation, a novel approach that softens learning targets based on the degree of image transform, leading to improved accuracy, robustness, and calibration in image classification models.

Contribution

The paper proposes a new soft augmentation method that generalizes traditional invariant transforms by adjusting learning targets, enhancing model performance and calibration.

Findings

Doubles top-1 accuracy boost with aggressive augmentation

Improves occlusion robustness by up to 4x

Halves calibration error (ECE)

Abstract

Modern neural networks are over-parameterized and thus rely on strong regularization such as data augmentation and weight decay to reduce overfitting and improve generalization. The dominant form of data augmentation applies invariant transforms, where the learning target of a sample is invariant to the transform applied to that sample. We draw inspiration from human visual classification studies and propose generalizing augmentation with invariant transforms to soft augmentation where the learning target softens non-linearly as a function of the degree of the transform applied to the sample: e.g., more aggressive image crop augmentations produce less confident learning targets. We demonstrate that soft targets allow for more aggressive data augmentation, offer more robust performance boosts, work with other augmentation policies, and interestingly, produce better calibrated models (since they are trained to be less confident on aggressively cropped/occluded examples). Combined with existing aggressive augmentation strategies, soft target 1) doubles the top-1 accuracy boost across Cifar-10, Cifar-100, ImageNet-1K, and ImageNet-V2, 2) improves model occlusion performance by up to $4\times$, and 3) halves the expected calibration error (ECE). Finally, we show that soft augmentation generalizes to self-supervised classification tasks. Code available at https://github.com/youngleox/soft_augmentation