DIB-OD: Preserving the Invariant Core for Robust Heterogeneous Graph Adaptation via Decoupled Information Bottleneck and Online Distillation

TL;DR

DIB-OD introduces a novel framework that preserves an invariant core in heterogeneous graph neural network pretraining, enhancing robustness and generalization across diverse domains by disentangling task-relevant knowledge from domain noise.

Contribution

The paper proposes a decoupled information bottleneck and online distillation approach that explicitly decomposes representations into invariant and redundant parts for better domain adaptation.

Findings

DIB-OD outperforms state-of-the-art methods in chemical, biological, and social network domains.

It effectively isolates a stable invariant core that improves cross-domain generalization.

The method demonstrates superior anti-forgetting capabilities during target-domain adaptation.

Abstract

Graph Neural Network pretraining is pivotal for leveraging unlabeled graph data. However, generalizing across heterogeneous domains remains a major challenge due to severe distribution shifts. Existing methods primarily focus on intra-domain patterns, failing to disentangle task-relevant invariant knowledge from domain-specific redundant noise, leading to negative transfer and catastrophic forgetting. To this end, we propose DIB-OD, a novel framework designed to preserve the invariant core for robust heterogeneous graph adaptation through a Decoupled Information Bottleneck and Online Distillation framework. Our core innovation is the explicit decomposition of representations into orthogonal invariant and redundant subspaces. By utilizing an Information Bottleneck teacher-student distillation mechanism and the Hilbert-Schmidt Independence Criterion, we isolate a stable invariant core that transcends domain boundaries. Furthermore, a self-adaptive semantic regularizer is introduced to protect this core from corruption during target-domain adaptation by dynamically gating label influence based on predictive confidence. Extensive experiments across chemical, biological, and social network domains demonstrate that DIB-OD significantly outperforms state-of-the-art methods, particularly in challenging inter-type domain transfers, showcasing superior generalization and anti-forgetting performance.