Graph Neural Network-Inspired Kernels for Gaussian Processes in Semi-Supervised Learning

TL;DR

This paper integrates graph neural network inductive biases into Gaussian processes by deriving new kernels inspired by GNNs, improving semi-supervised learning on graph-structured data with competitive performance and scalability.

Contribution

It introduces a novel method to incorporate GNN-inspired kernels into Gaussian processes, enhancing their performance on graph data and providing scalable inference techniques.

Findings

GNNs are equivalent to certain GPs when infinitely wide.

Proposed kernels improve classification and regression accuracy.

Scalable approximation methods enable efficient inference on large datasets.

Abstract

Gaussian processes (GPs) are an attractive class of machine learning models because of their simplicity and flexibility as building blocks of more complex Bayesian models. Meanwhile, graph neural networks (GNNs) emerged recently as a promising class of models for graph-structured data in semi-supervised learning and beyond. Their competitive performance is often attributed to a proper capturing of the graph inductive bias. In this work, we introduce this inductive bias into GPs to improve their predictive performance for graph-structured data. We show that a prominent example of GNNs, the graph convolutional network, is equivalent to some GP when its layers are infinitely wide; and we analyze the kernel universality and the limiting behavior in depth. We further present a programmable procedure to compose covariance kernels inspired by this equivalence and derive example kernels corresponding to several interesting members of the GNN family. We also propose a computationally efficient approximation of the covariance matrix for scalable posterior inference with large-scale data. We demonstrate that these graph-based kernels lead to competitive classification and regression performance, as well as advantages in computation time, compared with the respective GNNs.