Scalable Gaussian Processes with Latent Kronecker Structure

TL;DR

This paper introduces a scalable Gaussian process method that leverages latent Kronecker structures to efficiently handle large datasets with missing data, outperforming existing approaches in real-world applications.

Contribution

It proposes a novel latent Kronecker structure approach for Gaussian processes, enabling exact inference on large, incomplete datasets with improved computational efficiency.

Findings

Outperforms state-of-the-art sparse and variational GPs on large datasets

Handles missing data without losing Kronecker structure

Demonstrates effectiveness on datasets with up to five million examples

Abstract

Applying Gaussian processes (GPs) to very large datasets remains a challenge due to limited computational scalability. Matrix structures, such as the Kronecker product, can accelerate operations significantly, but their application commonly entails approximations or unrealistic assumptions. In particular, the most common path to creating a Kronecker-structured kernel matrix is by evaluating a product kernel on gridded inputs that can be expressed as a Cartesian product. However, this structure is lost if any observation is missing, breaking the Cartesian product structure, which frequently occurs in real-world data such as time series. To address this limitation, we propose leveraging latent Kronecker structure, by expressing the kernel matrix of observed values as the projection of a latent Kronecker product. In combination with iterative linear system solvers and pathwise conditioning, our method facilitates inference of exact GPs while requiring substantially fewer computational resources than standard iterative methods. We demonstrate that our method outperforms state-of-the-art sparse and variational GPs on real-world datasets with up to five million examples, including robotics, automated machine learning, and climate applications.