Discriminative Representation learning via Attention-Enhanced Contrastive Learning for Short Text Clustering

TL;DR

This paper introduces AECL, a novel short text clustering method that enhances contrastive learning with attention mechanisms to generate more discriminative representations and effectively address false negative issues.

Contribution

The paper proposes AECL, a new short text clustering approach that integrates attention-enhanced contrastive learning with pseudo-labeling to improve representation discriminability.

Findings

AECL outperforms state-of-the-art methods in short text clustering.

The attention mechanism improves sample similarity capture.

AECL effectively mitigates false negative separation issues.

Abstract

Contrastive learning has gained significant attention in short text clustering, yet it has an inherent drawback of mistakenly identifying samples from the same category as negatives and then separating them in the feature space (false negative separation), which hinders the generation of superior representations. To generate more discriminative representations for efficient clustering, we propose a novel short text clustering method, called Discriminative Representation learning via \textbf{A}ttention-\textbf{E}nhanced \textbf{C}ontrastive \textbf{L}earning for Short Text Clustering (\textbf{AECL}). The \textbf{AECL} consists of two modules which are the pseudo-label generation module and the contrastive learning module. Both modules build a sample-level attention mechanism to capture similarity relationships between samples and aggregate cross-sample features to generate consistent representations. Then, the former module uses the more discriminative consistent representation to produce reliable supervision information for assist clustering, while the latter module explores similarity relationships and consistent representations optimize the construction of positive samples to perform similarity-guided contrastive learning, effectively addressing the false negative separation issue. Experimental results demonstrate that the proposed \textbf{AECL} outperforms state-of-the-art methods. If the paper is accepted, we will open-source the code.