Short Text Clustering with Transformers

TL;DR

This paper explores the use of Transformer-based sentence embeddings combined with clustering algorithms to improve short text clustering, demonstrating that iterative classification enhances initial results.

Contribution

It introduces a novel approach using Transformer sentence vectors and iterative classification to advance short text clustering performance.

Findings

Transformer sentence embeddings improve clustering accuracy

Iterative classification further enhances clustering results

Pre-trained Transformer models are effective for short text clustering

Abstract

Recent techniques for the task of short text clustering often rely on word embeddings as a transfer learning component. This paper shows that sentence vector representations from Transformers in conjunction with different clustering methods can be successfully applied to address the task. Furthermore, we demonstrate that the algorithm of enhancement of clustering via iterative classification can further improve initial clustering performance with different classifiers, including those based on pre-trained Transformer language models.