Adversarial Label Invariant Graph Data Augmentations for Out-of-Distribution Generalization

TL;DR

This paper introduces RIA, an adversarial data augmentation method for out-of-distribution graph classification, improving generalization across different environments by preventing in-distribution collapse.

Contribution

It proposes a novel adversarial label-invariant augmentation technique combined with an optimization algorithm for OoD graph data, enhancing robustness against distribution shifts.

Findings

RIA achieves higher accuracy than baseline methods on synthetic and natural OoD graph classification tasks.

The method effectively prevents in-distribution collapse during training.

It can be integrated with existing OoD generalization approaches for covariate shift.

Abstract

Out-of-distribution (OoD) generalization occurs when representation learning encounters a distribution shift. This occurs frequently in practice when training and testing data come from different environments. Covariate shift is a type of distribution shift that occurs only in the input data, while the concept distribution stays invariant. We propose RIA - Regularization for Invariance with Adversarial training, a new method for OoD generalization under convariate shift. Motivated by an analogy to $Q$-learning, it performs an adversarial exploration for counterfactual data environments. These new environments are induced by adversarial label invariant data augmentations that prevent a collapse to an in-distribution trained learner. It works with many existing OoD generalization methods for covariate shift that can be formulated as constrained optimization problems. We develop an alternating gradient descent-ascent algorithm to solve the problem in the context of causally generated graph data, and perform extensive experiments on OoD graph classification for various kinds of synthetic and natural distribution shifts. We demonstrate that our method can achieve high accuracy compared with OoD baselines.