A Deep Dive into Deep Cluster
Ahmad Mustapha, Wael Khreich, Wasim Masr

TL;DR
This paper thoroughly analyzes DeepCluster, an unsupervised pretraining method for visual representations, revealing key factors affecting its performance and proposing practical hyperparameter guidelines for scalable training.
Contribution
It provides an in-depth understanding of DeepCluster's internal mechanisms, evaluates hyperparameter effects, and offers guidelines for effective and scalable unsupervised pretraining.
Findings
DeepCluster performance depends on initial filter quality and number of clusters.
Continuous clustering is not necessary for convergence.
Early stopping reduces training time without sacrificing performance.
Abstract
Deep Learning has demonstrated a significant improvement against traditional machine learning approaches in different domains such as image and speech recognition. Their success on benchmark datasets is transferred to the real-world through pretrained models by practitioners. Pretraining visual models using supervised learning requires a significant amount of expensive data annotation. To tackle this limitation, DeepCluster - a simple and scalable unsupervised pretraining of visual representations - has been proposed. However, the underlying work of the model is not yet well understood. In this paper, we analyze DeepCluster internals and exhaustively evaluate the impact of various hyperparameters over a wide range of values on three different datasets. Accordingly, we propose an explanation of why the algorithm works in practice. We also show that DeepCluster convergence and performance…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsDomain Adaptation and Few-Shot Learning · Multimodal Machine Learning Applications · Advanced Image and Video Retrieval Techniques
Methodsk-Means Clustering · DeepCluster · Early Stopping
