Deep Recurrent Gaussian Process with Variational Sparse Spectrum Approximation

TL;DR

This paper introduces two novel Deep Recurrent Gaussian Process models utilizing Sparse Spectrum approximations, enhancing uncertainty propagation and prediction accuracy in sequential data modeling, with applications in autonomous systems and new datasets.

Contribution

The paper presents two new Deep Recurrent Gaussian Process models based on Sparse Spectrum Gaussian Processes, incorporating variational inference and distributed optimization for improved accuracy and scalability.

Findings

Outperforms current state-of-the-art methods in prediction accuracy.

Successfully models uncertainty propagation through layers.

Introduces a new dataset for engine control called Emission.

Abstract

Modeling sequential data has become more and more important in practice. Some applications are autonomous driving, virtual sensors and weather forecasting. To model such systems so called recurrent models are used. In this article we introduce two new Deep Recurrent Gaussian Process (DRGP) models based on the Sparse Spectrum Gaussian Process (SSGP) and the improved variational version called Variational Sparse Spectrum Gaussian Process (VSSGP). We follow the recurrent structure given by an existing DRGP based on a specific sparse Nystr\"om approximation. Therefore, we also variationally integrate out the input-space and hence can propagate uncertainty through the layers. We can show that for the resulting lower bound an optimal variational distribution exists. Training is realized through optimizing the variational lower bound. Using Distributed Variational Inference (DVI), we can reduce the computational complexity. We improve over current state of the art methods in prediction accuracy for experimental data-sets used for their evaluation and introduce a new data-set for engine control, named Emission. Furthermore, our method can easily be adapted for unsupervised learning, e.g. the latent variable model and its deep version.