Fully Bayesian Autoencoders with Latent Sparse Gaussian Processes

TL;DR

This paper introduces a Bayesian autoencoder model with sparse Gaussian process priors on the latent space, capturing correlations between data samples and outperforming variational autoencoder-based methods.

Contribution

It presents a novel fully Bayesian sparse Gaussian process prior for autoencoders, enabling better modeling of data correlations in the latent space.

Findings

Consistently outperforms variational autoencoders in various tasks.

Effectively captures correlations between data samples.

Demonstrates strong results in representation learning and generative modeling.

Abstract

Autoencoders and their variants are among the most widely used models in representation learning and generative modeling. However, autoencoder-based models usually assume that the learned representations are i.i.d. and fail to capture the correlations between the data samples. To address this issue, we propose a novel Sparse Gaussian Process Bayesian Autoencoder (SGPBAE) model in which we impose fully Bayesian sparse Gaussian Process priors on the latent space of a Bayesian Autoencoder. We perform posterior estimation for this model via stochastic gradient Hamiltonian Monte Carlo. We evaluate our approach qualitatively and quantitatively on a wide range of representation learning and generative modeling tasks and show that our approach consistently outperforms multiple alternatives relying on Variational Autoencoders.