# Unbiased Stochastic Optimization for Gaussian Processes on Finite Dimensional RKHS

## TL;DR

This paper introduces exact stochastic inference algorithms for Gaussian Processes with kernels inducing finite or infinite dimensional RKHSs, improving convergence guarantees and performance under memory constraints.

## Contribution

The authors develop novel algorithms for unbiased stochastic inference in GPs with finite-dimensional RKHS kernels, extending to infinite dimensions with approximate methods.

## Key findings

- Outperforms existing methods in memory-limited scenarios
- Achieves better convergence to true marginal likelihood
- Effective for both finite and infinite dimensional RKHSs

## Abstract

Current methods for stochastic hyperparameter learning in Gaussian Processes (GPs) rely on approximations, such as computing biased stochastic gradients or using inducing points in stochastic variational inference. However, when using such methods we are not guaranteed to converge to a stationary point of the true marginal likelihood. In this work, we propose algorithms for exact stochastic inference of GPs with kernels that induce a Reproducing Kernel Hilbert Space (RKHS) of moderate finite dimension. Our approach can also be extended to infinite dimensional RKHSs at the cost of forgoing exactness. Both for finite and infinite dimensional RKHSs, our method achieves better experimental results than existing methods when memory resources limit the feasible batch size and the possible number of inducing points.