Gaussian process regression with log-linear scaling for common non-stationary kernels

TL;DR

This paper presents a fast, scalable algorithm for Gaussian process regression with non-stationary kernels, leveraging NUFFT techniques to improve efficiency and extend applicability beyond stationary kernels.

Contribution

The authors develop a novel $O(N ext{log}N)$ kernel matrix-vector multiplication method for non-stationary Gaussian processes using NUFFT, generalizing stationary kernel approaches.

Findings

Achieves near-optimal $O(N ext{log}N)$ complexity for kernel matrix-vector products.

Demonstrates improved scalability over existing methods in higher spatial dimensions.

Provides rigorous error analysis and practical validation through numerical experiments.

Abstract

We introduce a fast algorithm for Gaussian process regression in low dimensions, applicable to a widely-used family of non-stationary kernels. The non-stationarity of these kernels is induced by arbitrary spatially-varying vertical and horizontal scales. In particular, any stationary kernel can be accommodated as a special case, and we focus especially on the generalization of the standard Mat\'ern kernel. Our subroutine for kernel matrix-vector multiplications scales almost optimally as $O(N\log N)$, where $N$ is the number of regression points. Like the recently developed equispaced Fourier Gaussian process (EFGP) methodology, which is applicable only to stationary kernels, our approach exploits non-uniform fast Fourier transforms (NUFFTs). We offer a complete analysis controlling the approximation error of our method, and we validate the method's practical performance with numerical experiments. In particular we demonstrate improved scalability compared to to state-of-the-art rank-structured approaches in spatial dimension $d>1$.