Language-Assisted Image Clustering Guided by Discriminative Relational Signals and Adaptive Semantic Centers

TL;DR

This paper introduces a novel language-assisted image clustering framework that leverages discriminative relational signals and adaptive semantic centers, significantly improving clustering accuracy and interpretability across multiple datasets.

Contribution

It proposes a new LAIC method utilizing cross-modal relations and prompt-based semantic centers, addressing limitations of previous approaches.

Findings

Achieves 2.6% average improvement over state-of-the-art methods.

Produces highly interpretable semantic centers.

Demonstrates effectiveness on eight benchmark datasets.

Abstract

Language-Assisted Image Clustering (LAIC) augments the input images with additional texts with the help of vision-language models (VLMs) to promote clustering performance. Despite recent progress, existing LAIC methods often overlook two issues: (i) textual features constructed for each image are highly similar, leading to weak inter-class discriminability; (ii) the clustering step is restricted to pre-built image-text alignments, limiting the potential for better utilization of the text modality. To address these issues, we propose a new LAIC framework with two complementary components. First, we exploit cross-modal relations to produce more discriminative self-supervision signals for clustering, as it compatible with most VLMs training mechanisms. Second, we learn category-wise continuous semantic centers via prompt learning to produce the final clustering assignments. Extensive experiments on eight benchmark datasets demonstrate that our method achieves an average improvement of 2.6% over state-of-the-art methods, and the learned semantic centers exhibit strong interpretability. Code is available in the supplementary material.