Variational Auto Encoder Gradient Clustering

TL;DR

This paper explores how gradient ascent in the VAE latent space can improve clustering by enhancing cluster separation and aiding in determining the optimal number of clusters, with promising results on MNIST.

Contribution

It introduces a novel gradient-based data processing method within the VAE framework to improve clustering clarity and cluster number estimation.

Findings

Gradient processing leads to more distinct clusters.

Proposed method simplifies cluster number determination.

Baseline GMM on t-SNE achieves state-of-the-art MNIST results.

Abstract

Clustering using deep neural network models have been extensively studied in recent years. Among the most popular frameworks are the VAE and GAN frameworks, which learns latent feature representations of data through encoder / decoder neural net structures. This is a suitable base for clustering tasks, as the latent space often seems to effectively capture the inherent essence of data, simplifying its manifold and reducing noise. In this article, the VAE framework is used to investigate how probability function gradient ascent over data points can be used to process data in order to achieve better clustering. Improvements in classification is observed comparing with unprocessed data, although state of the art results are not obtained. Processing data with gradient descent however results in more distinct cluster separation, making it simpler to investigate suitable hyper parameter settings such as the number of clusters. We propose a simple yet effective method for investigating suitable number of clusters for data, based on the DBSCAN clustering algorithm, and demonstrate that cluster number determination is facilitated with gradient processing. As an additional curiosity, we find that our baseline model used for comparison; a GMM on a t-SNE latent space for a VAE structure with weight one on reconstruction during training (autoencoder), yield state of the art results on the MNIST data, to our knowledge not beaten by any other existing model.