Individual and Structural Graph Information Bottlenecks for Out-of-Distribution Generalization

TL;DR

This paper introduces IS-GIB, a unified framework that improves out-of-distribution graph generalization by removing irrelevant features and leveraging structural class relationships through information bottlenecks.

Contribution

It proposes the novel I-GIB and S-GIB methods, unifying individual and structural information bottlenecks for enhanced OOD graph learning.

Findings

IS-GIB outperforms existing methods on node- and graph-level OOD tasks.

The framework effectively discards spurious features caused by distribution shifts.

Structural correlations improve the robustness of graph representations.

Abstract

Out-of-distribution (OOD) graph generalization are critical for many real-world applications. Existing methods neglect to discard spurious or noisy features of inputs, which are irrelevant to the label. Besides, they mainly conduct instance-level class-invariant graph learning and fail to utilize the structural class relationships between graph instances. In this work, we endeavor to address these issues in a unified framework, dubbed Individual and Structural Graph Information Bottlenecks (IS-GIB). To remove class spurious feature caused by distribution shifts, we propose Individual Graph Information Bottleneck (I-GIB) which discards irrelevant information by minimizing the mutual information between the input graph and its embeddings. To leverage the structural intra- and inter-domain correlations, we propose Structural Graph Information Bottleneck (S-GIB). Specifically for a batch of graphs with multiple domains, S-GIB first computes the pair-wise input-input, embedding-embedding, and label-label correlations. Then it minimizes the mutual information between input graph and embedding pairs while maximizing the mutual information between embedding and label pairs. The critical insight of S-GIB is to simultaneously discard spurious features and learn invariant features from a high-order perspective by maintaining class relationships under multiple distributional shifts. Notably, we unify the proposed I-GIB and S-GIB to form our complementary framework IS-GIB. Extensive experiments conducted on both node- and graph-level tasks consistently demonstrate the superior generalization ability of IS-GIB. The code is available at https://github.com/YangLing0818/GraphOOD.