Uncertainty quantification for sparse spectral variational approximations in Gaussian process regression

TL;DR

This paper analyzes the frequentist properties of spectral variational sparse Gaussian process regression, providing guarantees and bounds on credible sets and inducing variables, with theoretical insights supported by numerical experiments.

Contribution

It offers new theoretical guarantees for spectral variational sparse Gaussian processes, including coverage and contraction rate bounds, and explores their implications for different priors.

Findings

Credible sets have quantifiable frequentist coverage.

Minimum inducing variables needed for optimal contraction rates.

Numerical results align with theoretical predictions.

Abstract

We investigate the frequentist guarantees of the variational sparse Gaussian process regression model. In the theoretical analysis, we focus on the variational approach with spectral features as inducing variables. We derive guarantees and limitations for the frequentist coverage of the resulting variational credible sets. We also derive sufficient and necessary lower bounds for the number of inducing variables required to achieve minimax posterior contraction rates. The implications of these results are demonstrated for different choices of priors. In a numerical analysis we consider a wider range of inducing variable methods and observe similar phenomena beyond the scope of our theoretical findings.