GraphBridge: Towards Arbitrary Transfer Learning in GNNs
Li Ju, Xingyi Yang, Qi Li, Xinchao Wang
TL;DR
GraphBridge is a framework that enables flexible transfer learning across different tasks and domains in GNNs without modifying the original models or graph structures, supporting diverse applications.
Contribution
It introduces a novel architecture with prediction heads and a bridging network, allowing knowledge transfer across heterogeneous GNN tasks and domains without structural changes.
Findings
Effective transfer across multiple GNN scenarios
Supports arbitrary output dimensions in transferred models
Validated on 16 diverse datasets
Abstract
Graph neural networks (GNNs) are conventionally trained on a per-domain, per-task basis. It creates a significant barrier in transferring the acquired knowledge to different, heterogeneous data setups. This paper introduces GraphBridge, a novel framework to enable knowledge transfer across disparate tasks and domains in GNNs, circumventing the need for modifications to task configurations or graph structures. Specifically, GraphBridge allows for the augmentation of any pre-trained GNN with prediction heads and a bridging network that connects the input to the output layer. This architecture not only preserves the intrinsic knowledge of the original model but also supports outputs of arbitrary dimensions. To mitigate the negative transfer problem, GraphBridge merges the source model with a concurrently trained model, thereby reducing the source bias when applied to the target domain. Our…
Peer Reviews
Decision·ICLR 2025 Poster
GraphBridge offers a versatile framework for efficient graph transfer learning, enabling pre-trained GNNs to tackle diverse tasks and domains without extensive reorganization. It achieves resource efficiency through two novel tuning methods, GSST and GMST. The framework supports both cross-level and cross-domain tasks, allowing scalable application across simple to complex scenarios. Extensive experiments demonstrate its adaptability and robustness.
Although the paper represents a good novel contribution, there are some issues, which are as follows: 1. The framework relies on high-quality pre-trained GNNs, which may not always be available or easy to obtain. 2. Some domain-specific nuances might still require additional adaptation, potentially limiting GraphBridge’s effectiveness in highly specialized applications. 3. Baselines are very less and not that recent may be due to the scope of the work.
- The paper addresses the challenging issue of cross-task and cross-domain transfer in GNNs. - The design of using side-tuning to mitigate negative transfer is rational. - The experiments are conducted across multiple scenarios and datasets to evaluate the performance.
- The technical novelty seems limited. The overall framework is a straightforward application of traditional pre-train finetune paradigm but lacks graph-specific innovative designs. - The paper directly uses MLP as a side network without comparing it to other networks, such as co-attention used by multimodal large language models in bridging two different modalities. - The improvements seem quite limited in some datasets, e.g., some results in Table 1. Can the authors elaborate more on the possi
Originality:GraphBridge introduces a novel approach to arbitrary transfer learning within Graph Neural Networks (GNNs), allowing models to be reused across a variety of tasks and domains without modifying task-specific configurations or graph structures. This approach is particularly original in that it combines prediction heads and a bridging network in a way that facilitates diverse task adaptation while preserving pretrained model knowledge. Quality:The paper presenting a comprehensive evalu
While the paper introduces two side-tuning techniques (GSST and GMST) to address negative transfer, it would be beneficial to delve deeper into when and why each approach is likely to succeed or fail. For instance, providing clearer conditions under which GSST vs. GMST is preferable would help practitioners select the appropriate method for specific tasks. Additionally, it might be useful to add a comparison with other common techniques for mitigating negative transfer, such as domain adaptation
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Graph Neural Networks · Machine Learning in Healthcare · Domain Adaptation and Few-Shot Learning