CR-VAE: Contrastive Regularization on Variational Autoencoders for Preventing Posterior Collapse

TL;DR

CR-VAE introduces a contrastive regularization technique to VAEs, enhancing mutual information between inputs and latent representations, thereby effectively preventing posterior collapse and improving the quality of learned representations.

Contribution

This paper presents a novel contrastive regularization method for VAEs that explicitly maximizes mutual information to prevent posterior collapse.

Findings

CR-VAE outperforms existing methods in preventing posterior collapse.

The contrastive regularization improves the mutual information between inputs and latent space.

Experimental results show enhanced representation quality on visual datasets.

Abstract

The Variational Autoencoder (VAE) is known to suffer from the phenomenon of \textit{posterior collapse}, where the latent representations generated by the model become independent of the inputs. This leads to degenerated representations of the input, which is attributed to the limitations of the VAE's objective function. In this work, we propose a novel solution to this issue, the Contrastive Regularization for Variational Autoencoders (CR-VAE). The core of our approach is to augment the original VAE with a contrastive objective that maximizes the mutual information between the representations of similar visual inputs. This strategy ensures that the information flow between the input and its latent representation is maximized, effectively avoiding posterior collapse. We evaluate our method on a series of visual datasets and demonstrate, that CR-VAE outperforms state-of-the-art approaches in preventing posterior collapse.