Nested Variational Compression in Deep Gaussian Processes

TL;DR

This paper introduces nested variational compression for deep Gaussian processes, enabling more scalable and parallelizable inference by extending the original variational compression approach.

Contribution

It extends variational compression with a nested approach, improving inference scalability and enabling stochastic variational inference in deep Gaussian processes.

Findings

Lower bound on likelihood can be parallelized

Method adapts easily for stochastic variational inference

Enhances scalability of deep Gaussian process inference

Abstract

Deep Gaussian processes provide a flexible approach to probabilistic modelling of data using either supervised or unsupervised learning. For tractable inference approximations to the marginal likelihood of the model must be made. The original approach to approximate inference in these models used variational compression to allow for approximate variational marginalization of the hidden variables leading to a lower bound on the marginal likelihood of the model [Damianou and Lawrence, 2013]. In this paper we extend this idea with a nested variational compression. The resulting lower bound on the likelihood can be easily parallelized or adapted for stochastic variational inference.