Accelerated training of Gaussian processes using banded square exponential covariances

TL;DR

This paper introduces a new method for faster Gaussian process training by approximating the covariance matrix with a banded structure, significantly reducing computational costs while maintaining accuracy.

Contribution

The authors develop a principled banded matrix approximation for SE covariance matrices, enabling more efficient GP training without sacrificing the structure of the original covariance.

Findings

The banded approximation reduces computational complexity in GP training.

The method preserves the covariance structure in 1D SE kernel cases.

Validation shows improved efficiency over variational sparse GP methods.

Abstract

We propose a novel approach to computationally efficient GP training based on the observation that square-exponential (SE) covariance matrices contain several off-diagonal entries extremely close to zero. We construct a principled procedure to eliminate those entries to produce a \emph{banded}-matrix approximation to the original covariance, whose inverse and determinant can be computed at a reduced computational cost, thus contributing to an efficient approximation to the likelihood function. We provide a theoretical analysis of the proposed method to preserve the structure of the original covariance in the 1D setting with SE kernel, and validate its computational efficiency against the variational free energy approach to sparse GPs.