UGSL: A Unified Framework for Benchmarking Graph Structure Learning
Bahare Fatemi, Sami Abu-El-Haija, Anton Tsitsulin, Mehran Kazemi,, Dustin Zelle, Neslihan Bulut, Jonathan Halcrow, Bryan Perozzi
TL;DR
This paper introduces UGSL, a unified benchmarking framework for graph structure learning in GNNs, enabling consistent comparison and analysis of various models' effectiveness and components.
Contribution
The paper proposes a unified framework that reformulates existing graph structure learning models, facilitating comprehensive benchmarking and analysis.
Findings
Unified framework enables consistent comparison of models
Analysis reveals strengths and weaknesses of different components
Benchmark results clarify effectiveness of various approaches
Abstract
Graph neural networks (GNNs) demonstrate outstanding performance in a broad range of applications. While the majority of GNN applications assume that a graph structure is given, some recent methods substantially expanded the applicability of GNNs by showing that they may be effective even when no graph structure is explicitly provided. The GNN parameters and a graph structure are jointly learned. Previous studies adopt different experimentation setups, making it difficult to compare their merits. In this paper, we propose a benchmarking strategy for graph structure learning using a unified framework. Our framework, called Unified Graph Structure Learning (UGSL), reformulates existing models into a single model. We implement a wide range of existing models in our framework and conduct extensive analyses of the effectiveness of different components in the framework. Our results provide a…
Peer Reviews
Decision·Submitted to ICLR 2024
1. The authors compare many structures, and their new method is clear. 2. Some ideas in the paper are helpful and can make people think more about this topic.
1. Lack of the baselines and large graph datasets: though the author did great jobs in searching on various architectures, the work itself is not enough for supporting a comprehensive paper. The used graph is relatively small. 2. Lack of the novelty: divide the training process of GSL is trivial, and and some tests are hard to understand because they are too similar.
- This paper systematically studies the problem of graph structure learning and compares different types of methods on different benchmark graphs. - Several insights have been provided based on the benchmarking studies which are potentially useful when selecting graph structure learning models .
- These insights provided in this paper rely on limited graph benchmark datasets and these datasets have limitations in 1) they are in relatively small sizes, e.g., 10k nodes. 2) Datasets with graphs reflect more homophily but heterophily. However, in practice, many graphs show the pattern of a mixture of both homophily and heterophily. 3) The number of datasets is relatively small, e.g., there are only 3 datasets w/ and w/o graphs respectively. 4) All the tasks are node classification (although
* A unified GSL framework is devised, which splits the GSL framework into four components, allowing for seamless substitution of different components. * The author conducted over 30,000 comparisons on each dataset and offered numerous valuable insights.
* The scope of the work is limited. The authors only utilize node features on all datasets, without using their original graph structures. This may limit the scope of the benchmark, especially when some GSL methods are specifically designed for refining graphs[2, 3, 5]. Moreover, only node classification is included. Other tasks such as graph-level tasks should also be taken into consideration. It would also be better to compare the time complexity and memory consumption of different components.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Graph Neural Networks · Recommender Systems and Techniques · Text and Document Classification Technologies