Bayesian Neighborhood Adaptation for Graph Neural Networks
Paribesh Regmi, Rui Li, Kishan KC
TL;DR
This paper introduces a Bayesian framework for adaptively determining the optimal neighborhood scope in graph neural networks, enhancing their performance and expressivity on various graph types.
Contribution
It models GNN message passing as a stochastic process using a beta process, enabling simultaneous scope inference and parameter optimization.
Findings
Achieves competitive or superior node classification accuracy.
Provides well-calibrated predictions across datasets.
Compatible with various GNN architectures.
Abstract
The neighborhood scope (i.e., number of hops) where graph neural networks (GNNs) aggregate information to characterize a node's statistical property is critical to GNNs' performance. Two-stage approaches, training and validating GNNs for every pre-specified neighborhood scope to search for the best setting, is a time-consuming task and tends to be biased due to the search space design. How to adaptively determine proper neighborhood scopes for the aggregation process for both homophilic and heterophilic graphs remains largely unexplored. We thus propose to model the GNNs' message-passing behavior on a graph as a stochastic process by treating the number of hops as a beta process. This Bayesian framework allows us to infer the most plausible neighborhood scope for message aggregation simultaneously with the optimization of GNN parameters. Our theoretical analysis shows that the scope…
Peer Reviews
Decision·Submitted to ICLR 2025
1. The idea of using a Bayesian model for inferring the neighborhood scope is novel. 2. The empirical analysis is quite thorough, with an evaluation of model performance, over-smoothing analysis, performance on homophilic and heterophilic datasets, as well as small and large datasets, and computation time. 3. The model performs well, or at least competitively, in comparison to other baselines, in various aspects like the ones mentioned above. 4. The theoretical analysis of over-smoothing prov
1. The literature review in Section 2 looks weak because the description of the works is more about the proposed algorithms, but not their relevance to the current work. As in, the descriptions look like "ABC work does XYZ", but their strengths, weaknesses and/or relevance to BNA are not discussed. For example, near line 137, we have "half-hop adds slow nodes at each edge" - it is neither clear what *slow nodes* are (not that it is relevant for this work), nor is it obvious what context this pie
The method proposed in this paper can adaptively apply to different base GNN models to capture long-range relationships.
The performance improvement is quite modest compared to the computational cost.
- Originality: The paper presents an original solution to the problem of manually tuning neighborhood scopes in GNNs by framing the task as a Bayesian inference problem, which is a novel approach. This differs from typical HPO&NAS methods and introduces a probabilistic perspective on neighborhood adaptation. - Quality: The theoretical analysis is rigorous, particularly in terms of expressivity, and the claims are backed by mathematical proofs. The experiments are thorough, spanning both homophi
- Diversity of baselines: It seems that the proposed method is evaluated as a plug-in trick for improving conventional GNNs. However, there have been lots of previous works that achieve adaptive depth in an efficient way (differentiable in some methods). It is necessary to include such works as the baselines here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Graph Neural Networks · Machine Learning and ELM · Graph Theory and Algorithms