Kernel Interpolation with Sparse Grids

TL;DR

This paper introduces the use of sparse grids within structured kernel interpolation to improve Gaussian process inference scalability in higher dimensions, offering a nearly linear time matrix-vector multiplication algorithm.

Contribution

The paper proposes integrating sparse grids into SKI, along with a novel fast matrix-vector multiplication method, enabling scalable GP inference in higher-dimensional spaces.

Findings

Sparse grids reduce the number of points needed for accurate interpolation.

The new algorithm achieves nearly linear time complexity for matrix-vector multiplication.

SKI with sparse grids maintains accuracy while scaling to higher dimensions.

Abstract

Structured kernel interpolation (SKI) accelerates Gaussian process (GP) inference by interpolating the kernel covariance function using a dense grid of inducing points, whose corresponding kernel matrix is highly structured and thus amenable to fast linear algebra. Unfortunately, SKI scales poorly in the dimension of the input points, since the dense grid size grows exponentially with the dimension. To mitigate this issue, we propose the use of sparse grids within the SKI framework. These grids enable accurate interpolation, but with a number of points growing more slowly with dimension. We contribute a novel nearly linear time matrix-vector multiplication algorithm for the sparse grid kernel matrix. Next, we describe how sparse grids can be combined with an efficient interpolation scheme based on simplices. With these changes, we demonstrate that SKI can be scaled to higher dimensions while maintaining accuracy.