Augmenting Variational Autoencoders with Sparse Labels: A Unified Framework for Unsupervised, Semi-(un)supervised, and Supervised Learning

TL;DR

This paper introduces a unified VAE framework that leverages sparse labels to improve both unsupervised and supervised learning, demonstrating that unlabeled data enhances classification and vice versa.

Contribution

A simple, extendable semi-supervised VAE architecture that improves classification and generative tasks by integrating sparse labels into the model.

Findings

Unlabeled data boosts unsupervised and classification performance.

Labeled data enhances both classification and generative tasks.

The proposed method outperforms purely supervised models in classification.

Abstract

We present a new flavor of Variational Autoencoder (VAE) that interpolates seamlessly between unsupervised, semi-supervised and fully supervised learning domains. We show that unlabeled datapoints not only boost unsupervised tasks, but also the classification performance. Vice versa, every label not only improves classification, but also unsupervised tasks. The proposed architecture is simple: A classification layer is connected to the topmost encoder layer, and then combined with the resampled latent layer for the decoder. The usual evidence lower bound (ELBO) loss is supplemented with a supervised loss target on this classification layer that is only applied for labeled datapoints. This simplicity allows for extending any existing VAE model to our proposed semi-supervised framework with minimal effort. In the context of classification, we found that this approach even outperforms a direct supervised setup.