Generalizing Across Domains via Cross-Gradient Training

TL;DR

CROSSGRAD is a novel training method that enhances domain generalization by using domain-guided perturbations and joint classifier training, outperforming traditional domain adversarial techniques without requiring domain adaptation at test time.

Contribution

The paper introduces CROSSGRAD, a new approach that leverages domain-guided data augmentation and joint classifier training to improve generalization across unseen domains.

Findings

CROSSGRAD outperforms generic perturbation methods in domain generalization.

Data augmentation with domain-guided perturbations improves stability and accuracy.

CROSSGRAD surpasses domain adversarial training in empirical evaluations.

Abstract

We present CROSSGRAD, a method to use multi-domain training data to learn a classifier that generalizes to new domains. CROSSGRAD does not need an adaptation phase via labeled or unlabeled data, or domain features in the new domain. Most existing domain adaptation methods attempt to erase domain signals using techniques like domain adversarial training. In contrast, CROSSGRAD is free to use domain signals for predicting labels, if it can prevent overfitting on training domains. We conceptualize the task in a Bayesian setting, in which a sampling step is implemented as data augmentation, based on domain-guided perturbations of input instances. CROSSGRAD parallelly trains a label and a domain classifier on examples perturbed by loss gradients of each other's objectives. This enables us to directly perturb inputs, without separating and re-mixing domain signals while making various distributional assumptions. Empirical evaluation on three different applications where this setting is natural establishes that (1) domain-guided perturbation provides consistently better generalization to unseen domains, compared to generic instance perturbation methods, and that (2) data augmentation is a more stable and accurate method than domain adversarial training.