Fast Gaussian process inference by exact Mat\'ern kernel decomposition

TL;DR

This paper introduces an exact and efficient kernel matrix-vector multiplication algorithm for Gaussian process inference, leveraging kernel decomposition into empirical distribution functions, especially effective for low-dimensional problems with large datasets.

Contribution

The paper presents a novel exact fast kernel MVM algorithm based on kernel decomposition, compatible with Matérn kernels, and incorporates a new method for fixed effects predictor functions.

Findings

Algorithm is highly effective for low-dimensional problems with hundreds of thousands of data points.

Exact kernel decomposition enables fast matrix-vector multiplication without approximation.

Implementation available at provided GitLab link.

Abstract

To speed up Gaussian process inference, a number of fast kernel matrix-vector multiplication (MVM) approximation algorithms have been proposed over the years. In this paper, we establish an exact fast kernel MVM algorithm based on exact kernel decomposition into weighted empirical cumulative distribution functions, compatible with a class of kernels which includes multivariate Mat\'ern kernels with half-integer smoothness parameter. This algorithm uses a divide-and-conquer approach, during which sorting outputs are stored in a data structure. We also propose a new algorithm to take into account some linear fixed effects predictor function. Our numerical experiments confirm that our algorithm is very effective for low-dimensional Gaussian process inference problems with hundreds of thousands of data points. An implementation of our algorithm is available at https://gitlab.com/warin/fastgaussiankernelregression.git.