Diffusing to the Top: Boost Graph Neural Networks with Minimal Hyperparameter Tuning

TL;DR

This paper introduces GNN-Diff, a diffusion-based framework that enhances GNN performance with minimal hyperparameter tuning, reducing computational costs while maintaining high stability and generalizability across various graph tasks.

Contribution

The paper proposes a novel graph-conditioned latent diffusion method to generate high-performing GNNs from sub-optimal hyperparameters, reducing tuning efforts.

Findings

GNN-Diff improves GNN performance with minimal hyperparameter tuning.

The method demonstrates high stability across multiple runs.

It generalizes well to unseen data across diverse graph tasks.

Abstract

Graph Neural Networks (GNNs) are proficient in graph representation learning and achieve promising performance on versatile tasks such as node classification and link prediction. Usually, a comprehensive hyperparameter tuning is essential for fully unlocking GNN's top performance, especially for complicated tasks such as node classification on large graphs and long-range graphs. This is usually associated with high computational and time costs and careful design of appropriate search spaces. This work introduces a graph-conditioned latent diffusion framework (GNN-Diff) to generate high-performing GNNs based on the model checkpoints of sub-optimal hyperparameters selected by a light-tuning coarse search. We validate our method through 166 experiments across four graph tasks: node classification on small, large, and long-range graphs, as well as link prediction. Our experiments involve 10 classic and state-of-the-art target models and 20 publicly available datasets. The results consistently demonstrate that GNN-Diff: (1) boosts the performance of GNNs with efficient hyperparameter tuning; and (2) presents high stability and generalizability on unseen data across multiple generation runs. The code is available at https://github.com/lequanlin/GNN-Diff.