B\'ezier Gaussian Processes for Tall and Wide Data

TL;DR

This paper introduces a novel Bezier kernel for Gaussian processes that enables efficient scaling to datasets with many features and observations, overcoming traditional limitations in high-dimensional settings.

Contribution

The paper proposes a Bezier kernel that allows exponential growth of summarizing variables with linear computational cost, improving Gaussian process scalability for wide data.

Findings

Kernel scales well with both tall and wide datasets

Empirical results show competitive predictive performance

Avoids matrix inverses and determinants in inference

Abstract

Modern approximations to Gaussian processes are suitable for "tall data", with a cost that scales well in the number of observations, but under-performs on ``wide data'', scaling poorly in the number of input features. That is, as the number of input features grows, good predictive performance requires the number of summarising variables, and their associated cost, to grow rapidly. We introduce a kernel that allows the number of summarising variables to grow exponentially with the number of input features, but requires only linear cost in both number of observations and input features. This scaling is achieved through our introduction of the B\'ezier buttress, which allows approximate inference without computing matrix inverses or determinants. We show that our kernel has close similarities to some of the most used kernels in Gaussian process regression, and empirically demonstrate the kernel's ability to scale to both tall and wide datasets.