High-Dimensional Gaussian Process Regression with Soft Kernel Interpolation

TL;DR

Soft Kernel Interpolation (SoftKI) is a scalable Gaussian Process regression method that combines structured kernel interpolation and variational inducing points to handle high-dimensional data efficiently.

Contribution

The paper introduces SoftKI, a novel approach that improves high-dimensional GP regression by integrating interpolation and variational techniques for better scalability.

Findings

SoftKI performs competitively with other GP methods at around 10 dimensions.

It overcomes SKI's limitations in high dimensions while maintaining flexibility.

SoftKI effectively approximates kernels using learned interpolation points.

Abstract

We introduce Soft Kernel Interpolation (SoftKI), a method that combines aspects of Structured Kernel Interpolation (SKI) and variational inducing point methods, to achieve scalable Gaussian Process (GP) regression on high-dimensional datasets. SoftKI approximates a kernel via softmax interpolation from a smaller number of interpolation points learned by optimizing a combination of the SoftKI marginal log-likelihood (MLL), and when needed, an approximate MLL for improved numerical stability. Consequently, it can overcome the dimensionality scaling challenges that SKI faces when interpolating from a dense and static lattice while retaining the flexibility of variational methods to adapt inducing points to the dataset. We demonstrate the effectiveness of SoftKI across various examples and show that it is competitive with other approximated GP methods when the data dimensionality is modest (around 10).