Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Trees

TL;DR

This paper introduces a novel framework for learning generalizable representations across diverse graph tasks using task-trees, enabling effective transfer and adaptation in graph neural networks.

Contribution

It proposes task-trees as a unified learning approach for cross-task generalization in graphs and introduces GIT, a graph foundation model demonstrating strong performance across multiple domains.

Findings

GIT achieves strong performance on over 30 graphs across five domains.

Pretraining on task-trees with a reconstruction objective enhances transferability.

GIT enables effective fine-tuning, in-context learning, and zero-shot generalization.

Abstract

Foundation models are pretrained on large-scale corpora to learn generalizable patterns across domains and tasks -- such as contours, textures, and edges in images, or tokens and sentences in text. In contrast, discovering such generalities in graph-structured data, especially across heterogeneous graph tasks, remains an open challenge. To address this, we propose a novel approach to cross-task generalization in graphs via task-trees, which serve as unified learning instances aligning node-, edge-, and graph-level tasks. We theoretically analyze the stability, transferability, and generalization properties of task-trees, showing that pretraining a graph neural network (GNN) on diverse task-trees with a reconstruction objective induces transferable knowledge. This enables efficient adaptation to downstream tasks with minimal fine-tuning. To validate our framework, we introduce Graph Generality Identifier on Task-Trees (GIT), a graph foundation model that demonstrates strong performance on over 30 graphs across five domains via fine-tuning, in-context learning, and zero-shot generalization. Code and data are available at https://github.com/Zehong-Wang/GIT.