Customized Multiple Clustering via Multi-Modal Subspace Proxy Learning

TL;DR

This paper introduces Multi-Sub, an innovative end-to-end multiple clustering method that leverages multi-modal subspace proxy learning with CLIP and GPT-4 to enable user-specific, customizable data grouping.

Contribution

The paper proposes a novel framework combining CLIP and GPT-4 for automatic proxy word generation, allowing flexible, user-oriented multiple clustering.

Findings

Outperforms existing methods on various datasets

Effectively aligns textual prompts with visual data

Enables customizable data clustering based on user preferences

Abstract

Multiple clustering aims to discover various latent structures of data from different aspects. Deep multiple clustering methods have achieved remarkable performance by exploiting complex patterns and relationships in data. However, existing works struggle to flexibly adapt to diverse user-specific needs in data grouping, which may require manual understanding of each clustering. To address these limitations, we introduce Multi-Sub, a novel end-to-end multiple clustering approach that incorporates a multi-modal subspace proxy learning framework in this work. Utilizing the synergistic capabilities of CLIP and GPT-4, Multi-Sub aligns textual prompts expressing user preferences with their corresponding visual representations. This is achieved by automatically generating proxy words from large language models that act as subspace bases, thus allowing for the customized representation of data in terms specific to the user's interests. Our method consistently outperforms existing baselines across a broad set of datasets in visual multiple clustering tasks. Our code is available at https://github.com/Alexander-Yao/Multi-Sub.