Diversity Boosted Learning for Domain Generalization with Large Number of Domains

TL;DR

This paper introduces DOMI, a diversity-boosted sampling framework using DPPs, to improve domain generalization across many domains by mitigating spurious correlations and enhancing robustness.

Contribution

It proposes a novel DPP-based sampling method that efficiently handles large numbers of domains, improving domain generalization performance.

Findings

DOMI improves robustness against spurious correlations.

Enhanced performance on rotated MNIST and iwildcam datasets.

Efficient sampling reduces computational costs.

Abstract

Machine learning algorithms minimizing the average training loss usually suffer from poor generalization performance due to the greedy exploitation of correlations among the training data, which are not stable under distributional shifts. It inspires various works for domain generalization (DG), where a series of methods, such as Causal Matching and FISH, work by pairwise domain operations. They would need $O(n^2)$ pairwise domain operations with $n$ domains, where each one is often highly expensive. Moreover, while a common objective in the DG literature is to learn invariant representations against domain-induced spurious correlations, we highlight the importance of mitigating spurious correlations caused by objects. Based on the observation that diversity helps mitigate spurious correlations, we propose a Diversity boosted twO-level saMplIng framework (DOMI) utilizing Determinantal Point Processes (DPPs) to efficiently sample the most informative ones among large number of domains. We show that DOMI helps train robust models against spurious correlations from both domain-side and object-side, substantially enhancing the performance of the backbone DG algorithms on rotated MNIST, rotated Fashion MNIST, and iwildcam datasets.