Subset-of-Data Variational Inference for Deep Gaussian-Processes Regression

TL;DR

This paper introduces a simplified training method for Deep Gaussian Processes by fixing inducing input locations to data subsets and sampling from a variational distribution, reducing complexity and maintaining performance.

Contribution

It proposes a novel approach that fixes inducing input locations to data subsets and samples from a variational distribution, simplifying and stabilizing DGP training.

Findings

Reduced training complexity and computational cost.

Maintained performance comparable to traditional methods.

Enhanced stability and sampling capability for DGPs.

Abstract

Deep Gaussian Processes (DGPs) are multi-layer, flexible extensions of Gaussian processes but their training remains challenging. Sparse approximations simplify the training but often require optimization over a large number of inducing inputs and their locations across layers. In this paper, we simplify the training by setting the locations to a fixed subset of data and sampling the inducing inputs from a variational distribution. This reduces the trainable parameters and computation cost without significant performance degradations, as demonstrated by our empirical results on regression problems. Our modifications simplify and stabilize DGP training while making it amenable to sampling schemes for setting the inducing inputs.