Phase transitions and optimal algorithms for semi-supervised classifications on graphs: from belief propagation to graph convolution network

TL;DR

This paper studies phase transitions in graph-based semi-supervised classification, proposing an optimal belief propagation algorithm and a graph convolution network, providing new benchmarks and insights into neural network limitations.

Contribution

It introduces a theoretically grounded belief propagation algorithm and a graph convolution network for semi-supervised classification, along with synthetic benchmarks for evaluation.

Findings

Identifies a phase transition limiting clustering performance.

Proposes a belief propagation algorithm that is asymptotically optimal.

Demonstrates BPGCN overcomes sparsity and overfitting issues in practice.

Abstract

We perform theoretical and algorithmic studies for the problem of clustering and semi-supervised classification on graphs with both pairwise relational information and single-point feature information, upon a joint stochastic block model for generating synthetic graphs with both edges and node features. Asymptotically exact analysis based on the Bayesian inference of the underlying model are conducted, using the cavity method in statistical physics. Theoretically, we identify a phase transition of the generative model, which puts fundamental limits on the ability of all possible algorithms in the clustering task of the underlying model. Algorithmically, we propose a belief propagation algorithm that is asymptotically optimal on the generative model, and can be further extended to a belief propagation graph convolution neural network (BPGCN) for semi-supervised classification on graphs. For the first time, well-controlled benchmark datasets with asymptotially exact properties and optimal solutions could be produced for the evaluation of graph convolution neural networks, and for the theoretical understanding of their strengths and weaknesses. In particular, on these synthetic benchmark networks we observe that existing graph convolution neural networks are subject to an sparsity issue and an ovefitting issue in practice, both of which are successfully overcome by our BPGCN. Moreover, when combined with classic neural network methods, BPGCN yields extraordinary classification performances on some real-world datasets that have never been achieved before.