Transferable Deep Clustering Model
Zheng Zhang, Liang Zhao
TL;DR
This paper introduces a transferable deep clustering model that adapts cluster centroids to new data distributions using an attention mechanism, enhancing transferability and efficiency over traditional methods.
Contribution
The paper proposes a novel attention-based module for deep clustering that allows automatic centroid adaptation across domains, improving transferability and performance.
Findings
Significantly improves clustering accuracy on target domains
Reduces computational cost compared to traditional methods
Theoretically more powerful than k-means and GMM
Abstract
Deep learning has shown remarkable success in the field of clustering recently. However, how to transfer a trained clustering model on a source domain to a target domain by leveraging the acquired knowledge to guide the clustering process remains challenging. Existing deep clustering methods often lack generalizability to new domains because they typically learn a group of fixed cluster centroids, which may not be optimal for the new domain distributions. In this paper, we propose a novel transferable deep clustering model that can automatically adapt the cluster centroids according to the distribution of data samples. Rather than learning a fixed set of centroids, our approach introduces a novel attention-based module that can adapt the centroids by measuring their relationship with samples. In addition, we theoretically show that our model is strictly more powerful than some classical…
Peer Reviews
Decision·Submitted to ICLR 2024
1. The proposed algorithm aims to tackle a new clustering problem, which is to obtain good clustering results on the new domain without any fine-tuning. 2. It provides some theoretical analysis of the proposed algorithm and the corresponding proofs. 3. It shows the experimental results on various datasets, including synthetic and real-world datasets.
1. How the scores for the other methods are computed in Table 1(synthetic dataset experiment)? For example, are the centroids of $k$-means method re-computed on the target domain? I wonder if the scores are obtained by just using the nearest neighbor rule with the centroids obtained on the source domain. If so, I don't think it is a fair comparison because the proposed algorithm re-computed their centroids on the target centroids. The centroids of $k$-means also can be easily updated without fin
1. This paper proposes a method to address the issue of domain transfer in the deep clustering and provides theoretical analysis. 2. The targeting problem is an inherent challenge of deep clustering. The paper provides an in-depth analysis of the domian shfit problem in Introduction section. 3. The paper has a well-organized structure, and the explanation of the algorithm design is relatively clear.
1. The method proposed may introduce an increased complexity to the model, potentially resulting in longer training times and higher computational demands. 2. The experiments were conducted on some small datasets. The limited size of the current dataset may hinder a comprehensive evaluation of the proposed method. 3. The methods compared in the experiments of this paper may not be the most relevant to the target problem. Therefore, the experiments may not prove that TDCM is SOTA. 4. This pape
1. The proposed model makes sense.
1. Is there any obvious advantage to using TDCM instead of using an unsupervised model on the target domain? Is it faster or does it have less cost? 2. How to decide the number of updating blocks L? 3. The experiment is too weak. 1) For the synthetic dataset, how about changing the number of centroids for the target domain? How about changing the size of each clustering? 2）Only two simple real-world datasets are used. A larger and more complicated dataset should be used. 3) For CIFAR-10, using o
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsDomain Adaptation and Few-Shot Learning