Variational Gaussian Processes: A Functional Analysis View

TL;DR

This paper offers a unified functional analysis perspective on variational Gaussian processes, clarifying the relationship between features, kernel ridge regression, and GP approximations.

Contribution

It introduces a Banach space framework for GPs, unifying various variational feature choices and linking them to kernel ridge regression.

Findings

Unified perspective clarifies feature selection in variational GPs

Connects variational GPs with kernel ridge regression

Provides theoretical insights into GP approximation methods

Abstract

Variational Gaussian process (GP) approximations have become a standard tool in fast GP inference. This technique requires a user to select variational features to increase efficiency. So far the common choices in the literature are disparate and lacking generality. We propose to view the GP as lying in a Banach space which then facilitates a unified perspective. This is used to understand the relationship between existing features and to draw a connection between kernel ridge regression and variational GP approximations.