Discrepancy Minimization in Domain Generalization with Generative Nearest Neighbors

TL;DR

This paper introduces GNNDM, a novel domain generalization method that minimizes discrepancy using generative nearest neighbors, providing theoretical guarantees and outperforming existing approaches on benchmark datasets.

Contribution

The paper proposes a discrepancy minimization approach with a generative nearest neighbor mechanism that does not require domain labels, improving generalization in domain shift scenarios.

Findings

Outperforms state-of-the-art DG methods on PACS and VLCS datasets.

Provides theoretical guarantees for discrepancy minimization.

Does not require domain labels, making it more practical.

Abstract

Domain generalization (DG) deals with the problem of domain shift where a machine learning model trained on multiple-source domains fail to generalize well on a target domain with different statistics. Multiple approaches have been proposed to solve the problem of domain generalization by learning domain invariant representations across the source domains that fail to guarantee generalization on the shifted target domain. We propose a Generative Nearest Neighbor based Discrepancy Minimization (GNNDM) method which provides a theoretical guarantee that is upper bounded by the error in the labeling process of the target. We employ a Domain Discrepancy Minimization Network (DDMN) that learns domain agnostic features to produce a single source domain while preserving the class labels of the data points. Features extracted from this source domain are learned using a generative model whose latent space is used as a sampler to retrieve the nearest neighbors for the target data points. The proposed method does not require access to the domain labels (a more realistic scenario) as opposed to the existing approaches. Empirically, we show the efficacy of our method on two datasets: PACS and VLCS. Through extensive experimentation, we demonstrate the effectiveness of the proposed method that outperforms several state-of-the-art DG methods.