Recyclable Gaussian Processes

TL;DR

This paper introduces a novel framework for recycling Gaussian process approximations, enabling efficient ensemble construction without revisiting data, applicable to various tasks and scalable to large datasets.

Contribution

We develop a framework that combines independent variational Gaussian process approximations into ensembles without data reprocessing, supporting diverse tasks and models.

Findings

Effective in large-scale distributed experiments

Comparable to exact inference models in accuracy

Supports heterogeneous regression and classification tasks

Abstract

We present a new framework for recycling independent variational approximations to Gaussian processes. The main contribution is the construction of variational ensembles given a dictionary of fitted Gaussian processes without revisiting any subset of observations. Our framework allows for regression, classification and heterogeneous tasks, i.e. mix of continuous and discrete variables over the same input domain. We exploit infinite-dimensional integral operators based on the Kullback-Leibler divergence between stochastic processes to re-combine arbitrary amounts of variational sparse approximations with different complexity, likelihood model and location of the pseudo-inputs. Extensive results illustrate the usability of our framework in large-scale distributed experiments, also compared with the exact inference models in the literature.