Scalable Multi-Output Gaussian Processes with Stochastic Variational Inference

TL;DR

This paper introduces a scalable stochastic variational inference method for Latent Variable Multi-Output Gaussian Processes, enabling efficient modeling of large output sets with reduced computational complexity.

Contribution

It proposes a novel stochastic variational inference technique for LV-MOGP that handles mini-batches over inputs and outputs, improving scalability.

Findings

Computational complexity per iteration becomes independent of the number of outputs.

The method enables efficient training on large multi-output datasets.

It maintains modeling flexibility with latent variables for outputs.

Abstract

The Multi-Output Gaussian Process is is a popular tool for modelling data from multiple sources. A typical choice to build a covariance function for a MOGP is the Linear Model of Coregionalization (LMC) which parametrically models the covariance between outputs. The Latent Variable MOGP (LV-MOGP) generalises this idea by modelling the covariance between outputs using a kernel applied to latent variables, one per output, leading to a flexible MOGP model that allows efficient generalization to new outputs with few data points. Computational complexity in LV-MOGP grows linearly with the number of outputs, which makes it unsuitable for problems with a large number of outputs. In this paper, we propose a stochastic variational inference approach for the LV-MOGP that allows mini-batches for both inputs and outputs, making computational complexity per training iteration independent of the number of outputs.