Random Feature Expansions for Deep Gaussian Processes

TL;DR

This paper introduces a scalable and practical approach for Deep Gaussian Processes using random feature expansions and stochastic variational inference, enabling effective modeling of complex data with uncertainty quantification.

Contribution

The authors propose a novel formulation of DGPs with random feature expansions and stochastic variational inference, significantly improving scalability and ease of construction.

Findings

Successfully scaled to datasets with 8 million observations

Handled DGP architectures with up to 30 hidden layers

Achieved state-of-the-art inference performance

Abstract

The composition of multiple Gaussian Processes as a Deep Gaussian Process (DGP) enables a deep probabilistic nonparametric approach to flexibly tackle complex machine learning problems with sound quantification of uncertainty. Existing inference approaches for DGP models have limited scalability and are notoriously cumbersome to construct. In this work, we introduce a novel formulation of DGPs based on random feature expansions that we train using stochastic variational inference. This yields a practical learning framework which significantly advances the state-of-the-art in inference for DGPs, and enables accurate quantification of uncertainty. We extensively showcase the scalability and performance of our proposal on several datasets with up to 8 million observations, and various DGP architectures with up to 30 hidden layers.