Mixture Model Auto-Encoders: Deep Clustering through Dictionary Learning

TL;DR

MixMate introduces a deep clustering architecture combining auto-encoders and mixture models, achieving competitive results with fewer parameters and improved interpretability by enforcing sparsity in the latent space.

Contribution

The paper presents MixMate, a novel auto-encoder based clustering method inspired by dictionary learning, that enhances interpretability and reduces parameter count.

Findings

Achieves competitive clustering performance on image datasets.

Uses significantly fewer parameters than existing deep clustering methods.

Enforces sparsity in latent representations for better interpretability.

Abstract

State-of-the-art approaches for clustering high-dimensional data utilize deep auto-encoder architectures. Many of these networks require a large number of parameters and suffer from a lack of interpretability, due to the black-box nature of the auto-encoders. We introduce Mixture Model Auto-Encoders (MixMate), a novel architecture that clusters data by performing inference on a generative model. Derived from the perspective of sparse dictionary learning and mixture models, MixMate comprises several auto-encoders, each tasked with reconstructing data in a distinct cluster, while enforcing sparsity in the latent space. Through experiments on various image datasets, we show that MixMate achieves competitive performance compared to state-of-the-art deep clustering algorithms, while using orders of magnitude fewer parameters.