Domain-invariant Feature Exploration for Domain Generalization

TL;DR

This paper introduces DIFEX, a novel method for domain generalization that explores both internal and mutual domain-invariant features using knowledge distillation and correlation alignment, leading to improved generalization across diverse datasets.

Contribution

The paper proposes a new framework, DIFEX, which captures internal and mutual invariance in features for better domain generalization, utilizing Fourier phase and correlation alignment.

Findings

DIFEX achieves state-of-the-art results on multiple benchmarks.

The method effectively captures high-level invariant features.

Increased feature diversity improves generalization performance.

Abstract

Deep learning has achieved great success in the past few years. However, the performance of deep learning is likely to impede in face of non-IID situations. Domain generalization (DG) enables a model to generalize to an unseen test distribution, i.e., to learn domain-invariant representations. In this paper, we argue that domain-invariant features should be originating from both internal and mutual sides. Internal invariance means that the features can be learned with a single domain and the features capture intrinsic semantics of data, i.e., the property within a domain, which is agnostic to other domains. Mutual invariance means that the features can be learned with multiple domains (cross-domain) and the features contain common information, i.e., the transferable features w.r.t. other domains. We then propose DIFEX for Domain-Invariant Feature EXploration. DIFEX employs a knowledge distillation framework to capture the high-level Fourier phase as the internally-invariant features and learn cross-domain correlation alignment as the mutually-invariant features. We further design an exploration loss to increase the feature diversity for better generalization. Extensive experiments on both time-series and visual benchmarks demonstrate that the proposed DIFEX achieves state-of-the-art performance.