GRAVER: Generative Graph Vocabularies for Robust Graph Foundation Models Fine-tuning

TL;DR

GRAVER introduces a generative vocabulary framework that enhances the robustness and efficiency of fine-tuning Graph Foundation Models, significantly improving few-shot classification performance across diverse graph domains.

Contribution

The paper proposes a novel generative graph vocabulary method that stabilizes and accelerates GFM fine-tuning through subgraph pattern extraction and graphon-based generative experts.

Findings

Outperforms 15 state-of-the-art baselines in robustness and efficiency

Effectively transfers subgraph patterns across domains

Enhances few-shot node and graph classification accuracy

Abstract

Inspired by the remarkable success of foundation models in language and vision, Graph Foundation Models (GFMs) hold significant promise for broad applicability across diverse graph tasks and domains. However, existing GFMs struggle with unstable few-shot fine-tuning, where both performance and adaptation efficiency exhibit significant fluctuations caused by the randomness in the support sample selection and structural discrepancies between the pre-trained and target graphs. How to fine-tune GFMs robustly and efficiently to enable trustworthy knowledge transfer across domains and tasks is the major challenge. In this paper, we propose GRAVER, a novel Generative gRAph VocabulariEs for Robust GFM fine-tuning framework that tackles the aforementioned instability via generative augmentations. Specifically, to identify transferable units, we analyze and extract key class-specific subgraph patterns by ego-graph disentanglement and validate their transferability both theoretically and empirically. To enable effective pre-training across diverse domains, we leverage a universal task template based on ego-graph similarity and construct graph vocabularies via graphon-based generative experts. To facilitate robust and efficient prompt fine-tuning, we grave the support samples with in-context vocabularies, where the lightweight MoE-CoE network attentively routes knowledge from source domains. Extensive experiments demonstrate the superiority of GRAVER over effectiveness, robustness, and efficiency on downstream few-shot node and graph classification tasks compared with 15 state-of-the-art baselines.