Non-Parametric Variational Inference with Graph Convolutional Networks for Gaussian Processes

TL;DR

This paper introduces a novel non-parametric variational inference method for Gaussian Processes that leverages Graph Convolutional Networks to efficiently approximate the posterior, especially for large datasets with local correlations.

Contribution

It proposes a neighborhood-based variational distribution combined with GCNs to improve inference speed and accuracy in Gaussian Processes, reducing parameter count and optimization iterations.

Findings

Faster inference compared to previous methods

More accurate results in large datasets

Efficient stochastic optimization with small batches

Abstract

Inference for GP models with non-Gaussian noises is computationally expensive when dealing with large datasets. Many recent inference methods approximate the posterior distribution with a simpler distribution defined on a small number of inducing points. The inference is accurate only when data points have strong correlation with these inducing points. In this paper, we consider the inference problem in a different direction: GP function values in the posterior are mostly correlated in short distance. We construct a variational distribution such that the inference for a data point considers only its neighborhood. With this construction, the variational lower bound is highly decomposible, hence we can run stochastic optimization with very small batches. We then train Graph Convolutional Networks as a reusable model to identify variational parameters for each data point. Model reuse greatly reduces the number of parameters and the number of iterations needed in optimization. The proposed method significantly speeds up the inference and often gets more accurate results than previous methods.