Hierarchical Inducing Point Gaussian Process for Inter-domain Observations

TL;DR

This paper introduces HIP-GP, a scalable hierarchical inducing point Gaussian process method for inter-domain observations, enabling high-accuracy approximations with millions of inducing points in low-dimensional settings.

Contribution

The paper presents HIP-GP, a novel scalable inter-domain GP inference method using grid-structured inducing points and new computational strategies.

Findings

Enables inference with millions of inducing points.

Provides a fast whitening strategy for GPs.

Introduces a new preconditioner for conjugate gradients.

Abstract

We examine the general problem of inter-domain Gaussian Processes (GPs): problems where the GP realization and the noisy observations of that realization lie on different domains. When the mapping between those domains is linear, such as integration or differentiation, inference is still closed form. However, many of the scaling and approximation techniques that our community has developed do not apply to this setting. In this work, we introduce the hierarchical inducing point GP (HIP-GP), a scalable inter-domain GP inference method that enables us to improve the approximation accuracy by increasing the number of inducing points to the millions. HIP-GP, which relies on inducing points with grid structure and a stationary kernel assumption, is suitable for low-dimensional problems. In developing HIP-GP, we introduce (1) a fast whitening strategy, and (2) a novel preconditioner for conjugate gradients which can be helpful in general GP settings. Our code is available at https: //github.com/cunningham-lab/hipgp.