Text-Free Multi-domain Graph Pre-training: Toward Graph Foundation Models

TL;DR

This paper introduces MDGPT, a text-free multi-domain graph pre-training framework that aligns diverse graph data using domain tokens and dual prompts, significantly improving performance across multiple datasets.

Contribution

The paper presents a novel text-free pre-training method for multi-domain graphs using domain tokens and dual prompts for better domain adaptation.

Findings

Outperforms previous methods by up to 37.9% on six datasets

Effectively aligns multi-domain graph features without textual descriptions

Enhances domain adaptation through dual prompt strategy

Abstract

Given the ubiquity of graph data, it is intriguing to ask: Is it possible to train a graph foundation model on a broad range of graph data across diverse domains? A major hurdle toward this goal lies in the fact that graphs from different domains often exhibit profoundly divergent characteristics. Although there have been some initial efforts in integrating multi-domain graphs for pre-training, they primarily rely on textual descriptions to align the graphs, limiting their application to text-attributed graphs. Moreover, different source domains may conflict or interfere with each other, and their relevance to the target domain can vary significantly. To address these issues, we propose MDGPT, a text free Multi-Domain Graph Pre-Training and adaptation framework designed to exploit multi-domain knowledge for graph learning. First, we propose a set of domain tokens to to align features across source domains for synergistic pre-training. Second, we propose a dual prompts, consisting of a unifying prompt and a mixing prompt, to further adapt the target domain with unified multi-domain knowledge and a tailored mixture of domain-specific knowledge. Finally, we conduct extensive experiments involving six public datasets to evaluate and analyze MDGPT, which outperforms prior art by up to 37.9%.