Joint Representation Learning and Clustering via Gradient-Based Manifold Optimization

TL;DR

This paper introduces a novel joint learning framework that combines dimensionality reduction and clustering using gradient-based manifold optimization, improving performance on high-dimensional data.

Contribution

It proposes a unified manifold learning approach that simultaneously optimizes dimension reduction and clustering parameters, which is a new integration in the field.

Findings

Outperforms popular clustering algorithms on benchmark datasets.

Effectively learns low-dimensional representations for clustering.

Demonstrates applicability to simulated and real image data.

Abstract

Clustering and dimensionality reduction have been crucial topics in machine learning and computer vision. Clustering high-dimensional data has been challenging for a long time due to the curse of dimensionality. For that reason, a more promising direction is the joint learning of dimension reduction and clustering. In this work, we propose a Manifold Learning Framework that learns dimensionality reduction and clustering simultaneously. The proposed framework is able to jointly learn the parameters of a dimension reduction technique (e.g. linear projection or a neural network) and cluster the data based on the resulting features (e.g. under a Gaussian Mixture Model framework). The framework searches for the dimension reduction parameters and the optimal clusters by traversing a manifold,using Gradient Manifold Optimization. The obtained The proposed framework is exemplified with a Gaussian Mixture Model as one simple but efficient example, in a process that is somehow similar to unsupervised Linear Discriminant Analysis (LDA). We apply the proposed method to the unsupervised training of simulated data as well as a benchmark image dataset (i.e. MNIST). The experimental results indicate that our algorithm has better performance than popular clustering algorithms from the literature.