Marginalising over Stationary Kernels with Bayesian Quadrature

TL;DR

This paper introduces a Bayesian Quadrature method for efficiently marginalising over Gaussian Process kernels, improving predictive accuracy and uncertainty calibration especially under limited computational resources.

Contribution

It presents a novel Bayesian Quadrature scheme that efficiently marginalises over kernel families using a kernel over kernels, enhancing Gaussian Process models.

Findings

More accurate predictions than state-of-the-art methods

Better calibrated uncertainty estimates

Effective under limited computational budgets

Abstract

Marginalising over families of Gaussian Process kernels produces flexible model classes with well-calibrated uncertainty estimates. Existing approaches require likelihood evaluations of many kernels, rendering them prohibitively expensive for larger datasets. We propose a Bayesian Quadrature scheme to make this marginalisation more efficient and thereby more practical. Through use of the maximum mean discrepancies between distributions, we define a kernel over kernels that captures invariances between Spectral Mixture (SM) Kernels. Kernel samples are selected by generalising an information-theoretic acquisition function for warped Bayesian Quadrature. We show that our framework achieves more accurate predictions with better calibrated uncertainty than state-of-the-art baselines, especially when given limited (wall-clock) time budgets.