Hybrid VAE: Improving Deep Generative Models using Partial Observations

TL;DR

Hybrid VAE leverages both labeled and unlabeled data to enhance deep generative models, acting as a regularizer and enabling high performance with limited labeled data across multiple datasets.

Contribution

The paper introduces Hybrid VAE, a novel framework that combines generative and discriminative components to effectively utilize unlabeled data for improved deep generative modeling.

Findings

Unlabeled data improves generative model performance with limited labeled data.

H-VAE achieves comparable results to fully-supervised models using fewer labeled samples.

Qualitative visualizations demonstrate the benefits of partial observations.

Abstract

Deep neural network models trained on large labeled datasets are the state-of-the-art in a large variety of computer vision tasks. In many applications, however, labeled data is expensive to obtain or requires a time consuming manual annotation process. In contrast, unlabeled data is often abundant and available in large quantities. We present a principled framework to capitalize on unlabeled data by training deep generative models on both labeled and unlabeled data. We show that such a combination is beneficial because the unlabeled data acts as a data-driven form of regularization, allowing generative models trained on few labeled samples to reach the performance of fully-supervised generative models trained on much larger datasets. We call our method Hybrid VAE (H-VAE) as it contains both the generative and the discriminative parts. We validate H-VAE on three large-scale datasets of different modalities: two face datasets: (MultiPIE, CelebA) and a hand pose dataset (NYU Hand Pose). Our qualitative visualizations further support improvements achieved by using partial observations.