Sequential Randomized Matrix Factorization for Gaussian Processes: Efficient Predictions and Hyper-parameter Optimization

TL;DR

This paper introduces a sequential randomized matrix factorization method for Gaussian Processes that enables efficient incremental predictions and hyper-parameter optimization, addressing computational bottlenecks in kernel matrix inversion.

Contribution

It formalizes a streaming approach for kernel matrix inversion using randomized factorization and extends it to optimize hyperparameters for certain kernel functions.

Findings

The proposed method improves computational efficiency over batch approaches.

It maintains high accuracy in incremental predictions.

Demonstrated effectiveness on two public datasets.

Abstract

This paper presents a sequential randomized lowrank matrix factorization approach for incrementally predicting values of an unknown function at test points using the Gaussian Processes framework. It is well-known that in the Gaussian processes framework, the computational bottlenecks are the inversion of the (regularized) kernel matrix and the computation of the hyper-parameters defining the kernel. The main contributions of this paper are two-fold. First, we formalize an approach to compute the inverse of the kernel matrix using randomized matrix factorization algorithms in a streaming scenario, i.e., data is generated incrementally over time. The metrics of accuracy and computational efficiency of the proposed method are compared against a batch approach based on use of randomized matrix factorization and an existing streaming approach based on approximating the Gaussian process by a finite set of basis vectors. Second, we extend the sequential factorization approach to a class of kernel functions for which the hyperparameters can be efficiently optimized. All results are demonstrated on two publicly available datasets.