FixCLR: Negative-Class Contrastive Learning for Semi-Supervised Domain Generalization

TL;DR

FixCLR introduces a contrastive learning approach that explicitly regularizes for domain invariance in semi-supervised domain generalization, improving out-of-distribution generalization with limited labels.

Contribution

The paper proposes FixCLR, a novel contrastive learning method that enhances domain invariance regularization in SSDG by leveraging pseudo-labels and a repelling loss, compatible with existing methods.

Findings

FixCLR improves SSDG performance across multiple datasets.

Combining FixCLR with other semi-supervised methods yields significant gains.

Pretrained models benefit more from FixCLR in domain generalization tasks.

Abstract

Semi-supervised domain generalization (SSDG) aims to solve the problem of generalizing to out-of-distribution data when only a few labels are available. Due to label scarcity, applying domain generalization methods often underperform. Consequently, existing SSDG methods combine semi-supervised learning methods with various regularization terms. However, these methods do not explicitly regularize to learn domains invariant representations across all domains, which is a key goal for domain generalization. To address this, we introduce FixCLR. Inspired by success in self-supervised learning, we change two crucial components to adapt contrastive learning for explicit domain invariance regularization: utilization of class information from pseudo-labels and using only a repelling term. FixCLR can also be added on top of most existing SSDG and semi-supervised methods for complementary performance improvements. Our research includes extensive experiments that have not been previously explored in SSDG studies. These experiments include benchmarking different improvements to semi-supervised methods, evaluating the performance of pretrained versus non-pretrained models, and testing on datasets with many domains. Overall, FixCLR proves to be an effective SSDG method, especially when combined with other semi-supervised methods.