Multiresolution Kernel Approximation for Gaussian Process Regression

TL;DR

This paper introduces Multiresolution Kernel Approximation (MKA), a novel, memory-efficient method for Gaussian process regression that effectively handles broad bandwidth kernels and scales to larger datasets.

Contribution

MKA is the first broad bandwidth kernel approximation algorithm that is memory efficient and directly approximates the kernel matrix, its inverse, and determinant.

Findings

MKA outperforms existing methods on large datasets.

It effectively approximates kernels with small length scales.

MKA enables scalable Gaussian process regression.

Abstract

Gaussian process regression generally does not scale to beyond a few thousands data points without applying some sort of kernel approximation method. Most approximations focus on the high eigenvalue part of the spectrum of the kernel matrix, $K$, which leads to bad performance when the length scale of the kernel is small. In this paper we introduce Multiresolution Kernel Approximation (MKA), the first true broad bandwidth kernel approximation algorithm. Important points about MKA are that it is memory efficient, and it is a direct method, which means that it also makes it easy to approximate $K^{-1}$ and $\mathop{\textrm{det}}(K)$.