ComStreamClust: a communicative multi-agent approach to text clustering in streaming data

TL;DR

This paper introduces ComStreamClust, a multi-agent, communicative clustering method for real-time detection and tracking of sub-topics in social media streams, demonstrated on COVID-19 and FA CUP datasets.

Contribution

It presents a novel parallelizable multi-agent clustering approach that effectively detects sub-topics in streaming social media data using semantic embeddings.

Findings

Outperforms existing clustering methods in accuracy

Efficiently handles multiple data points simultaneously

Effective in tracking evolving topics over time

Abstract

Topic detection is the task of determining and tracking hot topics in social media. Twitter is arguably the most popular platform for people to share their ideas with others about different issues. One such prevalent issue is the COVID-19 pandemic. Detecting and tracking topics on these kinds of issues would help governments and healthcare companies deal with this phenomenon. In this paper, we propose a novel, multi-agent, communicative clustering approach, so-called ComStreamClust for clustering sub-topics inside a broader topic, e.g., COVID-19. The proposed approach is parallelizable, and can simultaneously handle several data-point. The LaBSE sentence embedding is used to measure the semantic similarity between two tweets. ComStreamClust has been evaluated on two datasets: the COVID-19 and the FA CUP. The results obtained from ComStreamClust approve the effectiveness of the proposed approach when compared to existing methods.