SiamMM: A Mixture Model Perspective on Deep Unsupervised Learning

TL;DR

SiamMM introduces a mixture model framework to improve clustering-based unsupervised learning, achieving state-of-the-art results and revealing insights into label quality.

Contribution

This work connects clustering methods with classical mixture models, enhancing their effectiveness and introducing the SiamMM model for deep unsupervised learning.

Findings

SiamMM achieves state-of-the-art performance on benchmarks.

Learned clusters closely resemble ground truth labels.

The approach uncovers potential mislabeling in datasets.

Abstract

Recent studies have demonstrated the effectiveness of clustering-based approaches for self-supervised and unsupervised learning. However, the application of clustering is often heuristic, and the optimal methodology remains unclear. In this work, we establish connections between these unsupervised clustering methods and classical mixture models from statistics. Through this framework, we demonstrate significant enhancements to these clustering methods, leading to the development of a novel model named SiamMM. Our method attains state-of-the-art performance across various self-supervised learning benchmarks. Inspection of the learned clusters reveals a strong resemblance to unseen ground truth labels, uncovering potential instances of mislabeling.