Towards Unsupervised Domain Generalization

TL;DR

This paper introduces unsupervised domain generalization (UDG), a new approach that leverages unlabeled data for training models to improve their ability to generalize across unseen domains, reducing reliance on costly labeled data.

Contribution

The paper proposes DARLING, a novel method for UDG that effectively handles heterogeneity in unlabeled data and enhances model generalization, outperforming traditional pretraining methods like ImageNet.

Findings

DARLING improves generalization with unlabeled data.

DARLING outperforms ImageNet pretraining in low-label scenarios.

Unlabeled data can effectively substitute for labeled data in domain generalization.

Abstract

Domain generalization (DG) aims to help models trained on a set of source domains generalize better on unseen target domains. The performances of current DG methods largely rely on sufficient labeled data, which are usually costly or unavailable, however. Since unlabeled data are far more accessible, we seek to explore how unsupervised learning can help deep models generalize across domains. Specifically, we study a novel generalization problem called unsupervised domain generalization (UDG), which aims to learn generalizable models with unlabeled data and analyze the effects of pre-training on DG. In UDG, models are pretrained with unlabeled data from various source domains before being trained on labeled source data and eventually tested on unseen target domains. Then we propose a method named Domain-Aware Representation LearnING (DARLING) to cope with the significant and misleading heterogeneity within unlabeled pretraining data and severe distribution shifts between source and target data. Surprisingly we observe that DARLING can not only counterbalance the scarcity of labeled data but also further strengthen the generalization ability of models when the labeled data are insufficient. As a pretraining approach, DARLING shows superior or comparable performance compared with ImageNet pretraining protocol even when the available data are unlabeled and of a vastly smaller amount compared to ImageNet, which may shed light on improving generalization with large-scale unlabeled data.