Stick-Breaking Variational Autoencoders

TL;DR

This paper introduces the Stick-Breaking Variational Autoencoder (SB-VAE), a Bayesian nonparametric model that learns flexible, stochastic latent representations, often outperforming traditional Gaussian VAEs in discriminative tasks.

Contribution

It extends stochastic gradient variational inference to stick-breaking processes, creating a novel nonparametric VAE with adaptive latent dimensionality.

Findings

SB-VAE learns highly discriminative latent representations.

Semi-supervised SB-VAE outperforms Gaussian VAE.

Demonstrates effectiveness of nonparametric Bayesian methods in deep learning.

Abstract

We extend Stochastic Gradient Variational Bayes to perform posterior inference for the weights of Stick-Breaking processes. This development allows us to define a Stick-Breaking Variational Autoencoder (SB-VAE), a Bayesian nonparametric version of the variational autoencoder that has a latent representation with stochastic dimensionality. We experimentally demonstrate that the SB-VAE, and a semi-supervised variant, learn highly discriminative latent representations that often outperform the Gaussian VAE's.