Untapped Potential of Data Augmentation: A Domain Generalization Viewpoint

TL;DR

This paper examines the limitations of current data augmentation techniques from a domain generalization perspective, revealing that learned representations are less robust than expected and highlighting untapped potential for improvement.

Contribution

It introduces a domain generalization viewpoint to analyze data augmentation, showing that existing methods do not fully exploit their potential for robustness.

Findings

Learned representations are not as robust as expected.

Current augmentation methods do not fully utilize their potential.

Evidence of untapped potential in augmented data.

Abstract

Data augmentation is a popular pre-processing trick to improve generalization accuracy. It is believed that by processing augmented inputs in tandem with the original ones, the model learns a more robust set of features which are shared between the original and augmented counterparts. However, we show that is not the case even for the best augmentation technique. In this work, we take a Domain Generalization viewpoint of augmentation based methods. This new perspective allowed for probing overfitting and delineating avenues for improvement. Our exploration with the state-of-art augmentation method provides evidence that the learned representations are not as robust even towards distortions used during training. This suggests evidence for the untapped potential of augmented examples.