Learning Latent Superstructures in Variational Autoencoders for Deep Multidimensional Clustering

TL;DR

This paper introduces LTVAE, a variational autoencoder with a learned tree-structured superlatent variable, enabling multi-faceted clustering of high-dimensional data by capturing multiple natural partitions.

Contribution

The paper proposes LTVAE, a novel deep generative model that learns a superstructure of latent variables for multi-partition data clustering, extending traditional VAEs.

Findings

LTVAE can produce multiple meaningful data partitions.

The model reduces to Gaussian mixture VAEs with a single superlatent variable.

LTVAE effectively captures complex data structures.

Abstract

We investigate a variant of variational autoencoders where there is a superstructure of discrete latent variables on top of the latent features. In general, our superstructure is a tree structure of multiple super latent variables and it is automatically learned from data. When there is only one latent variable in the superstructure, our model reduces to one that assumes the latent features to be generated from a Gaussian mixture model. We call our model the latent tree variational autoencoder (LTVAE). Whereas previous deep learning methods for clustering produce only one partition of data, LTVAE produces multiple partitions of data, each being given by one super latent variable. This is desirable because high dimensional data usually have many different natural facets and can be meaningfully partitioned in multiple ways.