One Model for One Graph: A New Perspective for Pretraining with Cross-domain Graphs

TL;DR

This paper introduces a cross-domain pretraining framework for GNNs, enabling a single model to adapt across diverse graph domains by leveraging a bank of expert models and gating functions, improving generalization and performance.

Contribution

It proposes a novel 'one model for one graph' pretraining approach with expert models and gating, addressing domain-specific design challenges in GNNs.

Findings

Outperforms existing methods on link prediction tasks.

Achieves superior results in node classification across domains.

Demonstrates effective knowledge transfer with gating functions.

Abstract

Graph Neural Networks (GNNs) have emerged as a powerful tool to capture intricate network patterns, achieving success across different domains. However, existing GNNs require careful domain-specific architecture designs and training from scratch on each dataset, leading to an expertise-intensive process with difficulty in generalizing across graphs from different domains. Therefore, it can be hard for practitioners to infer which GNN model can generalize well to graphs from their domains. To address this challenge, we propose a novel cross-domain pretraining framework, "one model for one graph," which overcomes the limitations of previous approaches that failed to use a single GNN to capture diverse graph patterns across domains with significant gaps. Specifically, we pretrain a bank of expert models, with each one corresponding to a specific dataset. When inferring to a new graph, gating functions choose a subset of experts to effectively integrate prior model knowledge while avoiding negative transfer. Extensive experiments consistently demonstrate the superiority of our proposed method on both link prediction and node classification tasks.