Towards domain-invariant Self-Supervised Learning with Batch Styles Standardization

TL;DR

This paper introduces Batch Styles Standardization (BSS), a Fourier-based method that enhances domain-invariant self-supervised learning by standardizing image styles within batches, reducing reliance on domain labels and improving unseen domain generalization.

Contribution

The paper proposes BSS, a simple Fourier-based style standardization technique that integrates with SSL methods to improve domain-invariance without needing domain labels or architectures.

Findings

BSS significantly improves performance on unseen domains in UDG tasks.

BSS can be integrated with various SSL methods without domain-specific modifications.

Experimental results show BSS often outperforms existing UDG methods.

Abstract

In Self-Supervised Learning (SSL), models are typically pretrained, fine-tuned, and evaluated on the same domains. However, they tend to perform poorly when evaluated on unseen domains, a challenge that Unsupervised Domain Generalization (UDG) seeks to address. Current UDG methods rely on domain labels, which are often challenging to collect, and domain-specific architectures that lack scalability when confronted with numerous domains, making the current methodology impractical and rigid. Inspired by contrastive-based UDG methods that mitigate spurious correlations by restricting comparisons to examples from the same domain, we hypothesize that eliminating style variability within a batch could provide a more convenient and flexible way to reduce spurious correlations without requiring domain labels. To verify this hypothesis, we introduce Batch Styles Standardization (BSS), a relatively simple yet powerful Fourier-based method to standardize the style of images in a batch specifically designed for integration with SSL methods to tackle UDG. Combining BSS with existing SSL methods offers serious advantages over prior UDG methods: (1) It eliminates the need for domain labels or domain-specific network components to enhance domain-invariance in SSL representations, and (2) offers flexibility as BSS can be seamlessly integrated with diverse contrastive-based but also non-contrastive-based SSL methods. Experiments on several UDG datasets demonstrate that it significantly improves downstream task performances on unseen domains, often outperforming or rivaling with UDG methods. Finally, this work clarifies the underlying mechanisms contributing to BSS's effectiveness in improving domain-invariance in SSL representations and performances on unseen domain.