Does Invariant Graph Learning via Environment Augmentation Learn Invariance?

TL;DR

This paper investigates the challenge of learning invariant graph representations through environment augmentation, revealing fundamental limitations and proposing a new framework, GALA, that improves out-of-distribution generalization under minimal assumptions.

Contribution

The paper introduces minimal assumptions for feasible invariant graph learning and proposes GALA, a novel framework that uses an assistant model to identify invariant subgraphs for better OOD generalization.

Findings

GALA effectively identifies invariant subgraphs under minimal assumptions.

Experimental results on DrugOOD demonstrate GALA's superior OOD performance.

Augmenting environment information alone is insufficient without additional assumptions.

Abstract

Invariant graph representation learning aims to learn the invariance among data from different environments for out-of-distribution generalization on graphs. As the graph environment partitions are usually expensive to obtain, augmenting the environment information has become the de facto approach. However, the usefulness of the augmented environment information has never been verified. In this work, we find that it is fundamentally impossible to learn invariant graph representations via environment augmentation without additional assumptions. Therefore, we develop a set of minimal assumptions, including variation sufficiency and variation consistency, for feasible invariant graph learning. We then propose a new framework Graph invAriant Learning Assistant (GALA). GALA incorporates an assistant model that needs to be sensitive to graph environment changes or distribution shifts. The correctness of the proxy predictions by the assistant model hence can differentiate the variations in spurious subgraphs. We show that extracting the maximally invariant subgraph to the proxy predictions provably identifies the underlying invariant subgraph for successful OOD generalization under the established minimal assumptions. Extensive experiments on datasets including DrugOOD with various graph distribution shifts confirm the effectiveness of GALA.