Scalable Variational Gaussian Processes via Harmonic Kernel Decomposition

TL;DR

This paper presents a scalable variational Gaussian process method using harmonic kernel decomposition, enabling high-fidelity approximations with many inducing points, and demonstrating superior performance on image classification tasks.

Contribution

Introduces the harmonic kernel decomposition for scalable Gaussian processes, exploiting input symmetries for improved accuracy and efficiency.

Findings

Outperforms standard variational methods in scalability and accuracy

Achieves state-of-the-art results on CIFAR-10 among pure GP models

Effectively exploits input space symmetries like translations and reflections

Abstract

We introduce a new scalable variational Gaussian process approximation which provides a high fidelity approximation while retaining general applicability. We propose the harmonic kernel decomposition (HKD), which uses Fourier series to decompose a kernel as a sum of orthogonal kernels. Our variational approximation exploits this orthogonality to enable a large number of inducing points at a low computational cost. We demonstrate that, on a range of regression and classification problems, our approach can exploit input space symmetries such as translations and reflections, and it significantly outperforms standard variational methods in scalability and accuracy. Notably, our approach achieves state-of-the-art results on CIFAR-10 among pure GP models.