TL;DR
GraphProp introduces a novel training approach for graph foundation models that emphasizes structural invariants to enhance cross-domain generalization, especially in attribute-limited graph tasks.
Contribution
The paper proposes GraphProp, a two-phase training method focusing on structural invariants to improve graph model generalization across domains.
Findings
Outperforms existing models in supervised learning tasks.
Excels in few-shot learning scenarios.
Effective on graphs lacking node attributes.
Abstract
This work focuses on training graph foundation models (GFMs) that have strong generalization ability in graph-level tasks such as graph classification. Effective GFM training requires capturing information consistent across different domains. We discover that graph structures provide more consistent cross-domain information compared to node features and graph labels. However, traditional GFMs primarily focus on transferring node features from various domains into a unified representation space but often lack structural cross-domain generalization. To address this, we introduce GraphProp, which emphasizes structural generalization. The training process of GraphProp consists of two main phases. First, we train a structural GFM by predicting graph invariants. Since graph invariants are properties of graphs that depend only on the abstract structure, not on particular labellings or drawings…
Peer Reviews
Decision·Submitted to ICLR 2025
1. The paper addresses the GFM problem, which is highly significant across the entire field of graph analysis and presents a considerable challenge. However, the proposed approach is relatively simple given the complexity of the problem. 2. The paper provides a relatively clear related work section in the Appendix, which is helpful for readers outside this niche area to quickly gain foundational understanding. 3. The paper provides a clear explanation and comparison of various graph properties a
1. The primary concern with this paper lies in its misalignment between the proposed goal of achieving a GFM and the actual experiments conducted. The current experiments are limited to datasets designed for graph classification, focusing on a single task type, all at the graph level. This approach does not substantiate the broader scope implied by a GFM. We recommend either narrowing the scope explicitly to a GFM designed specifically for graph classification tasks or enhancing the experimental
S1. The proposed structural pre-training strategy is both intriguing and insightful to me. By pre-training the structural encoder with graph properties that have automatically obtainable labels, the proposed GraphProp enables the model to learn graph universal and hidden patterns. S2. The paper is well-organized and easy to follow. S3. The experiments are comprehensive, demonstrating the effectiveness of GraphProp.
W1. It's not clear how to apply GraphProp to zero-shot scenarios. Although I know that the used baseline (OFA) can be applied to zero-shot, as a reader, I am more curious to see how the proposed model can enhance the model performance in the zero-shot scenario. Especially how to use comprehensive training part in a zero-shot scenario. I suggest the authors provide a detailed explanation for this. W2. Figure 1 is missing a comparison with noise. From the prompts of TSGs and TAGs, there are more
The idea of training a universal structural representation is interesting. While most recent approaches try to combine graph learning ability and semantic learning ability, this work shows a promising direction to explore the possibility of training a model that understands graph structure well and injects that information into the semantic model. Overall, the paper is easy to follow, and it also provides a connection of the work to existing approaches.
- From the novelty and contribution perspective, while the idea of universal graph representation is interesting and promising, the proposed pipeline to acquire such an ability seems implausible to me. Specifically, - Can a graph transformer with the proposed positional encoding predict all proposed graph properties? Note that the graph transformer and the spectral encoding are constrained by 3-wl expressivity, and is their combination (theoretically) capable of predicting all the properties
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
