DiFiC: Your Diffusion Model Holds the Secret to Fine-Grained Clustering

TL;DR

DiFiC leverages conditional diffusion models and neighborhood similarity regularization to achieve superior fine-grained clustering performance, surpassing existing methods on multiple benchmarks.

Contribution

Introduces DiFiC, a novel fine-grained clustering approach using diffusion models and textual conditions, enhancing object semantics for improved clustering accuracy.

Findings

Outperforms state-of-the-art clustering methods on four benchmarks.

Effectively captures subtle differences between classes.

Demonstrates the potential of diffusion models in clustering tasks.

Abstract

Fine-grained clustering is a practical yet challenging task, whose essence lies in capturing the subtle differences between instances of different classes. Such subtle differences can be easily disrupted by data augmentation or be overwhelmed by redundant information in data, leading to significant performance degradation for existing clustering methods. In this work, we introduce DiFiC a fine-grained clustering method building upon the conditional diffusion model. Distinct from existing works that focus on extracting discriminative features from images, DiFiC resorts to deducing the textual conditions used for image generation. To distill more precise and clustering-favorable object semantics, DiFiC further regularizes the diffusion target and guides the distillation process utilizing neighborhood similarity. Extensive experiments demonstrate that DiFiC outperforms both state-of-the-art discriminative and generative clustering methods on four fine-grained image clustering benchmarks. We hope the success of DiFiC will inspire future research to unlock the potential of diffusion models in tasks beyond generation. The code will be released.