Learning Graph Foundation Models on Riemannian Graph-of-Graphs

TL;DR

R-GFM introduces a Riemannian Graph-of-Graphs model that adaptively captures multi-scale graph structures, improving generalization and performance across diverse graph tasks.

Contribution

It proposes a novel multi-scale, geometry-aware GFM framework using Riemannian manifolds to address fixed-scale limitations.

Findings

Achieves up to 49% relative improvement on downstream tasks.

Reduces structural domain generalization error.

Demonstrates state-of-the-art performance on various datasets.

Abstract

Graph foundation models (GFMs), pretrained on massive graph data, have transformed graph machine learning by supporting general-purpose reasoning across diverse graph tasks and domains. Existing GFMs pretrained with fixed-hop subgraph sampling impose a fixed receptive field, causing scale mismatch on diverse tasks, which often require heterogeneous and unknown structural contexts beyond a fixed sampling scale. We propose R-GFM, a Riemannian Graph-of-Graphs (GoG) based foundation model, that treats structural scale as a first-class citizen in modeling. R-GFM constructs a multi-scale GoG over-sampled subgraphs at different hop distances and learns geometry-adaptive representations from Riemannian manifolds. Theoretical analysis shows that R-GFM reduces structural domain generalization error compared to fixed-scale GFMs. Experiments on various datasets demonstrate that R-GFM achieves state-of-the-art performance, with up to a 49% relative improvement on downstream tasks. Our code is available at https://github.com/USTC-DataDarknessLab/R-GFM.