Combined Image Data Augmentations diminish the benefits of Adaptive Label Smoothing

TL;DR

This paper investigates how combining multiple aggressive image augmentations with adaptive label smoothing can reduce its effectiveness and harm model robustness, especially when diverse transformations are used.

Contribution

It extends adaptive label smoothing to various augmentations and analyzes its limitations with complex augmentation strategies like TrivialAugment.

Findings

Adaptive label smoothing improves regularization with strong augmentations like random erasing.

Its benefits diminish when combined with diverse transformations such as TrivialAugment.

Excessive label smoothing can reduce robustness to common corruptions.

Abstract

Soft augmentation regularizes the supervised learning process of image classifiers by reducing label confidence of a training sample based on the magnitude of random-crop augmentation applied to it. This paper extends this adaptive label smoothing framework to other types of aggressive augmentations beyond random-crop. Specifically, we demonstrate the effectiveness of the method for random erasing and noise injection data augmentation. Adaptive label smoothing permits stronger regularization via higher-intensity Random Erasing. However, its benefits vanish when applied with a diverse range of image transformations as in the state-of-the-art TrivialAugment method, and excessive label smoothing harms robustness to common corruptions. Our findings suggest that adaptive label smoothing should only be applied when the training data distribution is dominated by a limited, homogeneous set of image transformation types.