Semi-Supervised Graph Learning Meets Dimensionality Reduction

TL;DR

This paper explores how classical dimensionality reduction techniques like PCA, t-SNE, and UMAP can enhance semi-supervised graph neural networks by improving label propagation and clustering on benchmark datasets.

Contribution

It is the first comprehensive study to evaluate the impact of applying dimensionality reduction techniques to GNN inputs and outputs in semi-supervised learning.

Findings

Dimensionality reduction can improve label propagation in GNNs.

Applying PCA, t-SNE, UMAP enhances node clustering quality.

Certain conditions favor the effectiveness of these techniques.

Abstract

Semi-supervised learning (SSL) has recently received increased attention from machine learning researchers. By enabling effective propagation of known labels in graph-based deep learning (GDL) algorithms, SSL is poised to become an increasingly used technique in GDL in the coming years. However, there are currently few explorations in the graph-based SSL literature on exploiting classical dimensionality reduction techniques for improved label propagation. In this work, we investigate the use of dimensionality reduction techniques such as PCA, t-SNE, and UMAP to see their effect on the performance of graph neural networks (GNNs) designed for semi-supervised propagation of node labels. Our study makes use of benchmark semi-supervised GDL datasets such as the Cora and Citeseer datasets to allow meaningful comparisons of the representations learned by each algorithm when paired with a dimensionality reduction technique. Our comprehensive benchmarks and clustering visualizations quantitatively and qualitatively demonstrate that, under certain conditions, employing a priori and a posteriori dimensionality reduction to GNN inputs and outputs, respectively, can simultaneously improve the effectiveness of semi-supervised node label propagation and node clustering. Our source code is freely available on GitHub.