GC4NC: A Benchmark Framework for Graph Condensation on Node Classification with New Insights
Shengbo Gong, Juntong Ni, Noveen Sachdeva, Carl Yang, Wei Jin
TL;DR
This paper introduces GC4NC, a comprehensive benchmark framework for evaluating graph condensation methods in node classification, providing new insights into their behavior, effectiveness, and design choices.
Contribution
It presents a unified evaluation framework for graph condensation methods, enabling systematic comparison and analysis across multiple dimensions.
Findings
Condensed graphs can effectively preserve original graph information.
Design choices significantly impact GC method performance.
GC4NC reveals key factors influencing GC success.
Abstract
Graph condensation (GC) is an emerging technique designed to learn a significantly smaller graph that retains the essential information of the original graph. This condensed graph has shown promise in accelerating graph neural networks while preserving performance comparable to those achieved with the original, larger graphs. Additionally, this technique facilitates downstream applications like neural architecture search and deepens our understanding of redundancies in large graphs. Despite the rapid development of GC methods, particularly for node classification, a unified evaluation framework is still lacking to systematically compare different GC methods or clarify key design choices for improving their effectiveness. To bridge these gaps, we introduce \textbf{GC4NC}, a comprehensive framework for evaluating diverse GC methods on node classification across multiple dimensions…
Peer Reviews
Decision·ICLR 2025 Conference Withdrawn Submission
1. The framework, GC4NC, provides a standardized protocol that allows for the fair comparison of various GC methods on node classification across multiple critical dimensions such as performance, scalability, transferability. This systematic approach to evaluation is both comprehensive. 2. By being the first to systematically benchmark privacy preservation and denoising capabilities across various GC methods, the paper provides valuable insights that could lead to the development of more robust
1. The authors have adopted a fair evaluation protocol that includes training a 2-layer GCN with 256 hidden units on the reduced graph. While this setup might seem standardized, it could fail to represent the true predictive performance of the GCN model, as different hyperparameters such as the number of layers, dropout, and hidden units can significantly impact the node classification results on different datasets. In light of this, it is recommended that the authors consider adjusting the
1. The proposed benchmark compares graph condensation methods from many aspects besides predictive accuracy. 2. The paper is well-written and easy to follow in general. 3. Source codes are available to evaluate the effectiveness of the benchmark.
1. My major concern is how significant the proposed benchmark contributes to the graph machine learning community. Specifically, as explained in the paper, there are some earlier or concurrent works for graph benchmarking. As shown in Table 6 in the Appendix, these benchmarks seem to have their own advantages, e.g., this paper claims it has compared more methods, GC-Bench has considerably more datasets and broad tasks, and GCondenser has features such as continual learning and impact of validato
1. The categorizations of existing GC methods are well summarized. Also, an evaluation protocol and design choices are well-structured, e.g., structure-free vs. structure-based and graph property preservation. 2. This paper provides a framework in which researchers and developers can test various GC methods in a unified setting. On the framework, the authors conducted comprehensive experiments with 15+ methods on 7 datasets. 3. The authors discuss 11 observations in total through the experimen
1. While the paper provides experimental results from various aspects, some descriptions are insufficiently explained and insights are shallow. For example, 1. In lines 288-291, what factors of the datasets affect the performance? Why does Averaging achieve the best performance on Yelp? Why do Arxiv and Reddit have significant room for improvement? 2. In lines 350 and 351, the paper describes “GC methods can still perform well when the Instance Per Class (IPC) is as low as 1” but I foun
1. The paper addresses a significant gap by providing a unified and multi-dimensional evaluation protocol. It allows systematic comparisons and insights into GC methods, including under-explored aspects such as privacy preservation and denoising ability. 2. The findings offer novel insights, such as the trade-offs between GC performance and efficiency, the privacy benefits of GC methods, and how certain GC methods enable better transferability across GNN architectures. 3. The paper promises an o
1. From Table 6, in comparison with the existing works GCondenser [1] and GC-Bench [2], this paper's GC4NC is less comprehensive than GC-Bench. The advantage of GC4NC over GC-Bench lies in its inclusion of more Coreset & Sparsification methods; however, this is not a primary research focus for graph condensation. 2. Insufficient discussion of graph-level methods. Numerous studies focus on graph-level datasets, and it is essential to incorporate these works into the comparison framework. 3. Insuf
Code & Models
Videos
Taxonomy
TopicsSemantic Web and Ontologies · Model-Driven Software Engineering Techniques