Hub-VAE: Unsupervised Hub-based Regularization of Variational Autoencoders

TL;DR

Hub-VAE introduces a novel unsupervised regularization method for variational autoencoders using high-dimensional space hubs as exemplars, improving clustering, reconstruction, and generation.

Contribution

It proposes a data-driven, unsupervised regularization technique leveraging hubs as exemplars to enhance VAE performance in clustering and generative tasks.

Findings

Achieves superior cluster separability in embeddings

Provides accurate data reconstruction and generation

Outperforms baseline and state-of-the-art methods

Abstract

Exemplar-based methods rely on informative data points or prototypes to guide the optimization of learning algorithms. Such data facilitate interpretable model design and prediction. Of particular interest is the utility of exemplars in learning unsupervised deep representations. In this paper, we leverage hubs, which emerge as frequent neighbors in high-dimensional spaces, as exemplars to regularize a variational autoencoder and to learn a discriminative embedding for unsupervised down-stream tasks. We propose an unsupervised, data-driven regularization of the latent space with a mixture of hub-based priors and a hub-based contrastive loss. Experimental evaluation shows that our algorithm achieves superior cluster separability in the embedding space, and accurate data reconstruction and generation, compared to baselines and state-of-the-art techniques.