Faster Kernel Interpolation for Gaussian Processes

TL;DR

This paper introduces a novel approach to accelerate Gaussian Process inference by reducing per-iteration complexity to be independent of dataset size, enabling scalable analysis of massive datasets like weather radar data with over 100 million points.

Contribution

The authors reformulate structured kernel interpolation as a Bayesian linear regression problem, achieving per-iteration time complexity of O(m log m) after a single O(n) precomputation, significantly improving scalability.

Findings

Achieved speedups in inference time for large datasets.

Successfully applied method to weather radar data with over 100 million points.

Reduced per-iteration complexity to O(m log m), independent of dataset size n.

Abstract

A key challenge in scaling Gaussian Process (GP) regression to massive datasets is that exact inference requires computation with a dense n x n kernel matrix, where n is the number of data points. Significant work focuses on approximating the kernel matrix via interpolation using a smaller set of m inducing points. Structured kernel interpolation (SKI) is among the most scalable methods: by placing inducing points on a dense grid and using structured matrix algebra, SKI achieves per-iteration time of O(n + m log m) for approximate inference. This linear scaling in n enables inference for very large data sets; however the cost is per-iteration, which remains a limitation for extremely large n. We show that the SKI per-iteration time can be reduced to O(m log m) after a single O(n) time precomputation step by reframing SKI as solving a natural Bayesian linear regression problem with a fixed set of m compact basis functions. With per-iteration complexity independent of the dataset size n for a fixed grid, our method scales to truly massive data sets. We demonstrate speedups in practice for a wide range of m and n and apply the method to GP inference on a three-dimensional weather radar dataset with over 100 million points.