Optimal Exact Recovery in Semi-Supervised Learning: A Study of Spectral Methods and Graph Convolutional Networks

TL;DR

This paper investigates the limits of exact node classification in semi-supervised learning on the CSBM dataset, proposing optimal spectral methods and analyzing GCN performance near theoretical thresholds.

Contribution

It identifies the information-theoretical threshold for exact recovery in CSBM and introduces an optimal spectral estimator based on PCA, advancing understanding of spectral and GCN methods in semi-supervised learning.

Findings

Graph ridge regression and GCN can achieve the information threshold for exact recovery.

The optimal spectral estimator outperforms existing methods in synthetic CSBM datasets.

Feature learning enhances GCN performance near the theoretical recovery threshold.

Abstract

We delve into the challenge of semi-supervised node classification on the Contextual Stochastic Block Model (CSBM) dataset. Here, nodes from the two-cluster Stochastic Block Model (SBM) are coupled with feature vectors, which are derived from a Gaussian Mixture Model (GMM) that corresponds to their respective node labels. With only a subset of the CSBM node labels accessible for training, our primary objective becomes the accurate classification of the remaining nodes. Venturing into the transductive learning landscape, we, for the first time, pinpoint the information-theoretical threshold for the exact recovery of all test nodes in CSBM. Concurrently, we design an optimal spectral estimator inspired by Principal Component Analysis (PCA) with the training labels and essential data from both the adjacency matrix and feature vectors. We also evaluate the efficacy of graph ridge regression and Graph Convolutional Networks (GCN) on this synthetic dataset. Our findings underscore that graph ridge regression and GCN possess the ability to achieve the information threshold of exact recovery in a manner akin to the optimal estimator when using the optimal weighted self-loops. This highlights the potential role of feature learning in augmenting the proficiency of GCN, especially in the realm of semi-supervised learning.