Graph Semi-Supervised Learning for Point Classification on Data Manifolds

TL;DR

This paper introduces a graph semi-supervised learning framework for point classification on data manifolds, combining VAE-based manifold approximation, geometric graph construction, and GNNs, with theoretical analysis and empirical validation.

Contribution

It provides a theoretical analysis of the generalization properties of a manifold-based semi-supervised learning pipeline, including a novel resampling training procedure to improve generalization.

Findings

Generalization gap decreases with larger graphs under uniform sampling.

Resampling during training further reduces the generalization gap.

Empirical results on image benchmarks validate the approach.

Abstract

We propose a graph semi-supervised learning framework for classification tasks on data manifolds. Motivated by the manifold hypothesis, we model data as points sampled from a low-dimensional manifold $\mathcal{M} \subset \mathbb{R}^F$. The manifold is approximated in an unsupervised manner using a variational autoencoder (VAE), where the trained encoder maps data to embeddings that represent their coordinates in $\mathbb{R}^F$. A geometric graph is constructed with Gaussian-weighted edges inversely proportional to distances in the embedding space, transforming the point classification problem into a semi-supervised node classification task on the graph. This task is solved using a graph neural network (GNN). Our main contribution is a theoretical analysis of the statistical generalization properties of this data-to-manifold-to-graph pipeline. We show that, under uniform sampling from $\mathcal{M}$, the generalization gap of the semi-supervised task diminishes with increasing graph size, up to the GNN training error. Leveraging a training procedure which resamples a slightly larger graph at regular intervals during training, we then show that the generalization gap can be reduced even further, vanishing asymptotically. Finally, we validate our findings with numerical experiments on image classification benchmarks, demonstrating the empirical effectiveness of our approach.