Variational Auto-encoded Deep Gaussian Processes

TL;DR

This paper introduces a scalable deep Gaussian process model augmented with a recognition network, enabling efficient inference on large datasets for deep unsupervised learning and Bayesian optimization.

Contribution

It presents a novel variational framework that prevents parameter proliferation and allows distributed computation, making deep Gaussian processes scalable to large datasets.

Findings

Effective on large-scale datasets

Suitable for deep unsupervised learning

Applicable to deep Bayesian optimization

Abstract

We develop a scalable deep non-parametric generative model by augmenting deep Gaussian processes with a recognition model. Inference is performed in a novel scalable variational framework where the variational posterior distributions are reparametrized through a multilayer perceptron. The key aspect of this reformulation is that it prevents the proliferation of variational parameters which otherwise grow linearly in proportion to the sample size. We derive a new formulation of the variational lower bound that allows us to distribute most of the computation in a way that enables to handle datasets of the size of mainstream deep learning tasks. We show the efficacy of the method on a variety of challenges including deep unsupervised learning and deep Bayesian optimization.