Regularized Sparse Gaussian Processes

TL;DR

This paper introduces a regularization technique for sparse Gaussian processes that enhances inference and prediction, especially in latent variable models, by balancing data reconstruction and model approximation.

Contribution

It proposes a novel regularization method for sparse Gaussian processes, improving learning of inducing inputs and model performance in latent variable settings.

Findings

Regularization improves inference accuracy.

Enhanced prediction performance in latent models.

The approach is equivalent to variational inference on an empirical Bayes model.

Abstract

Gaussian processes are a flexible Bayesian nonparametric modelling approach that has been widely applied but poses computational challenges. To address the poor scaling of exact inference methods, approximation methods based on sparse Gaussian processes (SGP) are attractive. An issue faced by SGP, especially in latent variable models, is the inefficient learning of the inducing inputs, which leads to poor model prediction. We propose a regularization approach by balancing the reconstruction performance of data and the approximation performance of the model itself. This regularization improves both inference and prediction performance. We extend this regularization approach into latent variable models with SGPs and show that performing variational inference (VI) on those models is equivalent to performing VI on a related empirical Bayes model.