Revisiting the Sample Complexity of Sparse Spectrum Approximation of Gaussian Processes

TL;DR

This paper presents a scalable approximation method for Gaussian processes with provable guarantees, improving sample complexity analysis for sparse spectrum GPs and introducing an auto-encoding algorithm for better latent space disentanglement.

Contribution

It offers a new sample complexity analysis for sparse spectrum Gaussian processes and develops an auto-encoding algorithm for effective latent space disentanglement.

Findings

The method achieves low sample complexity under data disentangling conditions.

Validation on benchmarks shows promising results aligning with theoretical guarantees.

The approach improves scalability and approximation quality of Gaussian processes.

Abstract

We introduce a new scalable approximation for Gaussian processes with provable guarantees which hold simultaneously over its entire parameter space. Our approximation is obtained from an improved sample complexity analysis for sparse spectrum Gaussian processes (SSGPs). In particular, our analysis shows that under a certain data disentangling condition, an SSGP's prediction and model evidence (for training) can well-approximate those of a full GP with low sample complexity. We also develop a new auto-encoding algorithm that finds a latent space to disentangle latent input coordinates into well-separated clusters, which is amenable to our sample complexity analysis. We validate our proposed method on several benchmarks with promising results supporting our theoretical analysis.