Incorporating Supervised Domain Generalization into Data Augmentation

TL;DR

This paper introduces a supervised domain generalization approach with contrastive semantic alignment loss to enhance the robustness and training efficiency of data augmentation in deep learning models, especially under distribution shifts.

Contribution

It proposes a novel SDG-based method with CSA loss that can be integrated into existing data augmentation techniques to improve robustness and reduce training time.

Findings

Improves robustness of models against distribution shifts.

Reduces training epochs needed for effective data augmentation.

Enhances training efficiency without additional inference costs.

Abstract

With the increasing utilization of deep learning in outdoor settings, its robustness needs to be enhanced to preserve accuracy in the face of distribution shifts, such as compression artifacts. Data augmentation is a widely used technique to improve robustness, thanks to its ease of use and numerous benefits. However, it requires more training epochs, making it difficult to train large models with limited computational resources. To address this problem, we treat data augmentation as supervised domain generalization~(SDG) and benefit from the SDG method, contrastive semantic alignment~(CSA) loss, to improve the robustness and training efficiency of data augmentation. The proposed method only adds loss during model training and can be used as a plug-in for existing data augmentation methods. Experiments on the CIFAR-100 and CUB datasets show that the proposed method improves the robustness and training efficiency of typical data augmentations.