Reliable Pseudo-labeling via Optimal Transport with Attention for Short Text Clustering

TL;DR

This paper introduces POTA, a novel short text clustering framework that uses optimal transport and attention mechanisms to generate reliable pseudo-labels, improving discriminative representation learning and clustering accuracy.

Contribution

POTA integrates an attention mechanism with optimal transport to produce reliable pseudo-labels for short text clustering, handling data imbalance and enhancing discriminative features.

Findings

POTA outperforms state-of-the-art clustering methods.

The attention mechanism captures semantic relationships effectively.

Adaptive estimation of cluster distributions improves handling of imbalanced datasets.

Abstract

Short text clustering has gained significant attention in the data mining community. However, the limited valuable information contained in short texts often leads to low-discriminative representations, increasing the difficulty of clustering. This paper proposes a novel short text clustering framework, called Reliable \textbf{P}seudo-labeling via \textbf{O}ptimal \textbf{T}ransport with \textbf{A}ttention for Short Text Clustering (\textbf{POTA}), that generate reliable pseudo-labels to aid discriminative representation learning for clustering. Specially, \textbf{POTA} first implements an instance-level attention mechanism to capture the semantic relationships among samples, which are then incorporated as a semantic consistency regularization term into an optimal transport problem. By solving this OT problem, we can yield reliable pseudo-labels that simultaneously account for sample-to-sample semantic consistency and sample-to-cluster global structure information. Additionally, the proposed OT can adaptively estimate cluster distributions, making \textbf{POTA} well-suited for varying degrees of imbalanced datasets. Then, we utilize the pseudo-labels to guide contrastive learning to generate discriminative representations and achieve efficient clustering. Extensive experiments demonstrate \textbf{POTA} outperforms state-of-the-art methods. The code is available at: \href{https://github.com/YZH0905/POTA-STC/tree/main}{https://github.com/YZH0905/POTA-STC/tree/main}.