Feature Alignment and Restoration for Domain Generalization and Adaptation

TL;DR

This paper introduces FAR, a unified framework that combines feature alignment and restoration to improve domain generalization and adaptation by maintaining high discrimination and reducing domain discrepancy.

Contribution

The paper proposes a novel FAR framework that integrates feature alignment with a feature restoration process and a dual ranking entropy loss for better domain generalization and adaptation.

Findings

FAR achieves state-of-the-art results on multiple benchmarks.

The dual ranking entropy loss effectively disentangles task-relevant features.

FAR improves both generalization and discrimination in domain transfer tasks.

Abstract

For domain generalization (DG) and unsupervised domain adaptation (UDA), cross domain feature alignment has been widely explored to pull the feature distributions of different domains in order to learn domain-invariant representations. However, the feature alignment is in general task-ignorant and could result in degradation of the discrimination power of the feature representation and thus hinders the high performance. In this paper, we propose a unified framework termed Feature Alignment and Restoration (FAR) to simultaneously ensure high generalization and discrimination power of the networks for effective DG and UDA. Specifically, we perform feature alignment (FA) across domains by aligning the moments of the distributions of attentively selected features to reduce their discrepancy. To ensure high discrimination, we propose a Feature Restoration (FR) operation to distill task-relevant features from the residual information and use them to compensate for the aligned features. For better disentanglement, we enforce a dual ranking entropy loss constraint in the FR step to encourage the separation of task-relevant and task-irrelevant features. Extensive experiments on multiple classification benchmarks demonstrate the high performance and strong generalization of our FAR framework for both domain generalization and unsupervised domain adaptation.