Adaptive Methods for Aggregated Domain Generalization

TL;DR

This paper introduces a novel domain-adaptive method for domain generalization that clusters training data into pseudo-domains without using domain labels, achieving state-of-the-art results and providing theoretical guarantees.

Contribution

The paper proposes a clustering-based domain generalization approach that operates without domain labels, advancing privacy-preserving and large-scale learning applications.

Findings

Achieves state-of-the-art performance on multiple benchmarks.

Provides theoretical guarantees for domain generalization with pseudo-domains.

Effective even on large-scale datasets.

Abstract

Domain generalization involves learning a classifier from a heterogeneous collection of training sources such that it generalizes to data drawn from similar unknown target domains, with applications in large-scale learning and personalized inference. In many settings, privacy concerns prohibit obtaining domain labels for the training data samples, and instead only have an aggregated collection of training points. Existing approaches that utilize domain labels to create domain-invariant feature representations are inapplicable in this setting, requiring alternative approaches to learn generalizable classifiers. In this paper, we propose a domain-adaptive approach to this problem, which operates in two steps: (a) we cluster training data within a carefully chosen feature space to create pseudo-domains, and (b) using these pseudo-domains we learn a domain-adaptive classifier that makes predictions using information about both the input and the pseudo-domain it belongs to. Our approach achieves state-of-the-art performance on a variety of domain generalization benchmarks without using domain labels whatsoever. Furthermore, we provide novel theoretical guarantees on domain generalization using cluster information. Our approach is amenable to ensemble-based methods and provides substantial gains even on large-scale benchmark datasets. The code can be found at: https://github.com/xavierohan/AdaClust_DomainBed