Tracing Topic Transitions with Temporal Graph Clusters

TL;DR

This paper introduces an unsupervised graph-based framework to track and analyze the evolution of topics on Twitter over a two-week period, combining clustering algorithms and transition modeling.

Contribution

It presents a novel approach using Markov Clustering and transition modeling to identify and validate topic evolution on Twitter data.

Findings

Effective identification of sub-topic clusters over time

Transition flows align with human annotations

Framework captures dynamic topic changes

Abstract

Twitter serves as a data source for many Natural Language Processing (NLP) tasks. It can be challenging to identify topics on Twitter due to continuous updating data stream. In this paper, we present an unsupervised graph based framework to identify the evolution of sub-topics within two weeks of real-world Twitter data. We first employ a Markov Clustering Algorithm (MCL) with a node removal method to identify optimal graph clusters from temporal Graph-of-Words (GoW). Subsequently, we model the clustering transitions between the temporal graphs to identify the topic evolution. Finally, the transition flows generated from both computational approach and human annotations are compared to ensure the validity of our framework.