Exploiting Structure for Fast Kernel Learning

TL;DR

This paper introduces two scalable methods for exact Gaussian process inference on large datasets with missing data, leveraging Kronecker algebra for efficiency, and demonstrates their effectiveness on climate and video reconstruction tasks.

Contribution

It presents two novel scalable methods for exact GP inference on large, incomplete datasets using Kronecker algebra, with one method inferring missing responses prior to training.

Findings

Successfully performed exact GP inference on 3.7 million spatial-temporal points.

Achieved scalable inference on a video dataset with 1 billion points.

Demonstrated significant computational efficiency and low memory usage.

Abstract

We propose two methods for exact Gaussian process (GP) inference and learning on massive image, video, spatial-temporal, or multi-output datasets with missing values (or "gaps") in the observed responses. The first method ignores the gaps using sparse selection matrices and a highly effective low-rank preconditioner is introduced to accelerate computations. The second method introduces a novel approach to GP training whereby response values are inferred on the gaps before explicitly training the model. We find this second approach to be greatly advantageous for the class of problems considered. Both of these novel approaches make extensive use of Kronecker matrix algebra to design massively scalable algorithms which have low memory requirements. We demonstrate exact GP inference for a spatial-temporal climate modelling problem with 3.7 million training points as well as a video reconstruction problem with 1 billion points.