Towards Graph Foundation Models: Training on Knowledge Graphs Enables Transferability to General Graphs

TL;DR

This paper introduces SCR, a graph reasoning framework trained on knowledge graphs that generalizes well across various graph tasks and domains, reducing the need for fine-tuning.

Contribution

The paper presents a unified training framework on knowledge graphs with a novel semantic-conditioned message passing mechanism for broad graph task generalization.

Findings

SCR outperforms existing models on 38 diverse datasets

Significant performance improvements in inductive reasoning tasks

Effective across node, link, and graph-level tasks

Abstract

Inspired by the success of large language models, there is a trend toward developing graph foundation models to conduct diverse downstream tasks in various domains. However, current models often require extra fine-tuning to apply their learned structural and semantic representations to new graphs, which limits their versatility. Recent breakthroughs in zero-shot inductive reasoning on knowledge graphs (KGs), offer us a new perspective on extending KG reasoning to general graph applications. In this paper, we introduce SCR, a unified graph reasoning framework designed to train on knowledge graphs and effectively generalize across a wide range of graph tasks and domains. We begin by designing the task-specific KG structures to establish a unified topology for different task formats. Then we propose semantic-conditioned message passing, a novel mechanism addressing the inherent semantic isolation in traditional KG reasoning, by jointly modeling structural and semantic invariance patterns in graph representations. To demonstrate the effectiveness, we evaluate the inductive reasoning capability of SCR using 38 diverse graph datasets, covering node-level, link-level, and graph-level tasks across multiple domains. Our results show substantial performance gains over existing foundation models and supervised baselines, highlighting the efficacy and adaptability of our approach.