Bayesian Semi-supervised Learning with Graph Gaussian Processes

TL;DR

This paper introduces a Gaussian process-based Bayesian semi-supervised learning method for graphs that is data-efficient, competitive with state-of-the-art neural networks, and effective in low-label scenarios without needing validation sets.

Contribution

It presents a novel graph Gaussian process model that outperforms neural networks in semi-supervised and active learning tasks, with a Bayesian interpretation and scalable implementation.

Findings

Outperforms state-of-the-art graph neural networks in benchmark tests.

Excels in active learning scenarios with scarce labels.

Does not require validation data for early stopping.

Abstract

We propose a data-efficient Gaussian process-based Bayesian approach to the semi-supervised learning problem on graphs. The proposed model shows extremely competitive performance when compared to the state-of-the-art graph neural networks on semi-supervised learning benchmark experiments, and outperforms the neural networks in active learning experiments where labels are scarce. Furthermore, the model does not require a validation data set for early stopping to control over-fitting. Our model can be viewed as an instance of empirical distribution regression weighted locally by network connectivity. We further motivate the intuitive construction of the model with a Bayesian linear model interpretation where the node features are filtered by an operator related to the graph Laplacian. The method can be easily implemented by adapting off-the-shelf scalable variational inference algorithms for Gaussian processes.