GNUMAP: A Parameter-Free Approach to Unsupervised Dimensionality Reduction via Graph Neural Networks

TL;DR

GNUMAP is a new parameter-free graph neural network method that improves unsupervised node embedding quality for dimensionality reduction across diverse datasets, addressing hyperparameter sensitivity issues.

Contribution

It introduces GNUMAP, a robust, parameter-free approach combining UMAP and GNNs, and provides the first comprehensive benchmarking of unsupervised node embedding techniques.

Findings

GNUMAP outperforms existing GNN embedding methods across datasets.

Current methods are highly sensitive to hyperparameters.

GNUMAP is effective on synthetic, citation, and biomedical data.

Abstract

With the proliferation of Graph Neural Network (GNN) methods stemming from contrastive learning, unsupervised node representation learning for graph data is rapidly gaining traction across various fields, from biology to molecular dynamics, where it is often used as a dimensionality reduction tool. However, there remains a significant gap in understanding the quality of the low-dimensional node representations these methods produce, particularly beyond well-curated academic datasets. To address this gap, we propose here the first comprehensive benchmarking of various unsupervised node embedding techniques tailored for dimensionality reduction, encompassing a range of manifold learning tasks, along with various performance metrics. We emphasize the sensitivity of current methods to hyperparameter choices -- highlighting a fundamental issue as to their applicability in real-world settings where there is no established methodology for rigorous hyperparameter selection. Addressing this issue, we introduce GNUMAP, a robust and parameter-free method for unsupervised node representation learning that merges the traditional UMAP approach with the expressivity of the GNN framework. We show that GNUMAP consistently outperforms existing state-of-the-art GNN embedding methods in a variety of contexts, including synthetic geometric datasets, citation networks, and real-world biomedical data -- making it a simple but reliable dimensionality reduction tool.