Scalable Deep Unsupervised Clustering with Concrete GMVAEs

TL;DR

This paper introduces a continuous relaxation approach for Gaussian Mixture VAEs, significantly reducing training time for clustering tasks without sacrificing quality, demonstrated on CIFAR-100.

Contribution

It presents a novel continuous relaxation technique for discrete latent variables in VAEs, enabling scalable and efficient clustering.

Findings

Training time reduced from 47 hours to less than 6 hours on CIFAR-100.

No negative impact on clustering quality observed.

Method scalable to large datasets with many clusters.

Abstract

Discrete random variables are natural components of probabilistic clustering models. A number of VAE variants with discrete latent variables have been developed. Training such methods requires marginalizing over the discrete latent variables, causing training time complexity to be linear in the number clusters. By applying a continuous relaxation to the discrete variables in these methods we can achieve a reduction in the training time complexity to be constant in the number of clusters used. We demonstrate that in practice for one such method, the Gaussian Mixture VAE, the use of a continuous relaxation has no negative effect on the quality of the clustering but provides a substantial reduction in training time, reducing training time on CIFAR-100 with 20 clusters from 47 hours to less than 6 hours.