Self-supervised Document Clustering Based on BERT with Data Augment

TL;DR

This paper introduces a self-supervised contrastive learning method based on BERT with data augmentation, significantly improving text clustering performance, especially for short texts, and approaching supervised learning accuracy.

Contribution

It proposes a novel self-supervised contrastive learning framework with unsupervised data augmentation for effective document clustering.

Findings

SCL outperforms existing unsupervised clustering methods.

FCL achieves near-supervised performance.

UDA enhances clustering accuracy for short texts.

Abstract

Contrastive learning is a promising approach to unsupervised learning, as it inherits the advantages of well-studied deep models without a dedicated and complex model design. In this paper, based on bidirectional encoder representations from transformers, we propose self-supervised contrastive learning (SCL) as well as few-shot contrastive learning (FCL) with unsupervised data augmentation (UDA) for text clustering. SCL outperforms state-of-the-art unsupervised clustering approaches for short texts and those for long texts in terms of several clustering evaluation measures. FCL achieves performance close to supervised learning, and FCL with UDA further improves the performance for short texts.