Variational Orthogonal Features
David R. Burt, Carl Edward Rasmussen, Mark van der Wilk

TL;DR
This paper introduces a new class of variational features for stationary kernels that enable more efficient Gaussian process inference by reducing computational costs while maintaining accuracy.
Contribution
It proposes a construction of variational features applicable to any stationary kernel, significantly reducing inference complexity in Gaussian processes.
Findings
Unbiased ELBO estimator computed in ( ilde{N}T+M^2T) time
Approximate estimator computed in ( ilde{N}T+MT) time
Analysis of approximation impact on inference quality
Abstract
Sparse stochastic variational inference allows Gaussian process models to be applied to large datasets. The per iteration computational cost of inference with this method is where is the number of points in a minibatch and is the number of `inducing features', which determine the expressiveness of the variational family. Several recent works have shown that for certain priors, features can be defined that remove the cost of computing a minibatch estimate of an evidence lower bound (ELBO). This represents a significant computational savings when . We present a construction of features for any stationary prior kernel that allow for computation of an unbiased estimator to the ELBO using Monte Carlo samples in and in with an additional…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGaussian Processes and Bayesian Inference · Advanced Multi-Objective Optimization Algorithms · Control Systems and Identification
MethodsGaussian Process
