Unleash Graph Neural Networks from Heavy Tuning

TL;DR

This paper introduces GNN-Diff, a diffusion-based framework that generates high-performing Graph Neural Networks directly from limited tuning data, reducing the need for extensive hyperparameter search and improving generalization.

Contribution

The paper presents a novel diffusion framework for GNNs that bypasses heavy tuning and outperforms traditional search methods, enhancing efficiency and model quality.

Findings

GNN-Diff outperforms grid search in GNN performance.

Reduces computational costs of GNN tuning.

Produces higher-quality GNNs than general neural network diffusion methods.

Abstract

Graph Neural Networks (GNNs) are deep-learning architectures designed for graph-type data, where understanding relationships among individual observations is crucial. However, achieving promising GNN performance, especially on unseen data, requires comprehensive hyperparameter tuning and meticulous training. Unfortunately, these processes come with high computational costs and significant human effort. Additionally, conventional searching algorithms such as grid search may result in overfitting on validation data, diminishing generalization accuracy. To tackle these challenges, we propose a graph conditional latent diffusion framework (GNN-Diff) to generate high-performing GNNs directly by learning from checkpoints saved during a light-tuning coarse search. Our method: (1) unleashes GNN training from heavy tuning and complex search space design; (2) produces GNN parameters that outperform those obtained through comprehensive grid search; and (3) establishes higher-quality generation for GNNs compared to diffusion frameworks designed for general neural networks.