Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)

TL;DR

This paper introduces SKI, a flexible framework for scalable Gaussian processes that improves efficiency and scalability through kernel interpolation, leading to the development of KISS-GP, which is faster and more adaptable for large datasets.

Contribution

The paper presents SKI, a unifying framework for kernel interpolation in GPs, and introduces KISS-GP, a scalable method that leverages local cubic interpolation and algebraic structures for efficiency.

Findings

KISS-GP achieves O(n) inference time and storage.

KISS-GP outperforms inducing point methods in scalability.

KISS-GP enables fast kernel learning and natural sound modeling.

Abstract

We introduce a new structured kernel interpolation (SKI) framework, which generalises and unifies inducing point methods for scalable Gaussian processes (GPs). SKI methods produce kernel approximations for fast computations through kernel interpolation. The SKI framework clarifies how the quality of an inducing point approach depends on the number of inducing (aka interpolation) points, interpolation strategy, and GP covariance kernel. SKI also provides a mechanism to create new scalable kernel methods, through choosing different kernel interpolation strategies. Using SKI, with local cubic kernel interpolation, we introduce KISS-GP, which is 1) more scalable than inducing point alternatives, 2) naturally enables Kronecker and Toeplitz algebra for substantial additional gains in scalability, without requiring any grid data, and 3) can be used for fast and expressive kernel learning. KISS-GP costs O(n) time and storage for GP inference. We evaluate KISS-GP for kernel matrix approximation, kernel learning, and natural sound modelling.