Transformed Latent Variable Multi-Output Gaussian Processes

TL;DR

The paper introduces T-LVMOGP, a scalable multi-output Gaussian process framework that captures complex dependencies in high-dimensional output spaces using neural network embeddings and variational inference.

Contribution

It proposes a novel deep kernel construction with output-specific latent variables, enabling scalable and expressive multi-output Gaussian processes.

Findings

Outperforms baselines in climate modeling with 10,000+ outputs

Achieves better predictive accuracy and efficiency on spatial transcriptomics data

Effectively scales to high-dimensional output spaces using stochastic variational inference

Abstract

Multi-Output Gaussian Processes (MOGPs) provide a principled probabilistic framework for modelling correlated outputs but face scalability bottlenecks when applied to datasets with high-dimensional output spaces. To maintain tractability, existing methods typically resort to restrictive assumptions, such as employing low-rank or sum-of-separable kernels, which can limit expressiveness. We propose the Transformed Latent Variable MOGP (T-LVMOGP), a novel framework that scales MOGPs to a massive number of outputs while preserving the capacity to capture meaningful inter-output dependencies. T-LVMOGP constructs a flexible multi-output deep kernel by mapping inputs and output-specific latent variables into an embedding space using a Lipschitz-regularised neural network. Combined with stochastic variational inference, our model effectively scales to high-dimensional output settings. Across diverse benchmarks, including climate modelling with over 10,000 outputs and zero-inflated spatial transcriptomics data, T-LVMOGP outperforms baselines in both predictive accuracy and computational efficiency.