Semantic Data Augmentation based Distance Metric Learning for Domain Generalization

TL;DR

This paper introduces a novel domain generalization method that uses implicit semantic augmentation in feature space with a distance metric learning loss on logits, achieving state-of-the-art results without extra networks.

Contribution

It proposes a new approach leveraging logits for domain generalization, avoiding auxiliary networks and reducing computational costs.

Findings

Achieves state-of-the-art performance on three benchmarks.

Demonstrates the effectiveness of using logits for implicit augmentation.

Provides theoretical analysis supporting the use of logits for distance approximation.

Abstract

Domain generalization (DG) aims to learn a model on one or more different but related source domains that could be generalized into an unseen target domain. Existing DG methods try to prompt the diversity of source domains for the model's generalization ability, while they may have to introduce auxiliary networks or striking computational costs. On the contrary, this work applies the implicit semantic augmentation in feature space to capture the diversity of source domains. Concretely, an additional loss function of distance metric learning (DML) is included to optimize the local geometry of data distribution. Besides, the logits from cross entropy loss with infinite augmentations is adopted as input features for the DML loss in lieu of the deep features. We also provide a theoretical analysis to show that the logits can approximate the distances defined on original features well. Further, we provide an in-depth analysis of the mechanism and rational behind our approach, which gives us a better understanding of why leverage logits in lieu of features can help domain generalization. The proposed DML loss with the implicit augmentation is incorporated into a recent DG method, that is, Fourier Augmented Co-Teacher framework (FACT). Meanwhile, our method also can be easily plugged into various DG methods. Extensive experiments on three benchmarks (Digits-DG, PACS and Office-Home) have demonstrated that the proposed method is able to achieve the state-of-the-art performance.