Online Density-Based Clustering for Real-Time Narrative Evolution Monitorin

TL;DR

This paper explores replacing batch clustering with online density-based algorithms like DenStream to improve real-time social media narrative monitoring, balancing scalability, cluster quality, and interpretability.

Contribution

It introduces an online clustering approach tailored for large-scale, multilingual social media data streams, enhancing real-time narrative analysis capabilities.

Findings

DenStream outperforms other online methods in overall performance.

Trade-offs exist between temporal stability and narrative coherence.

Online clustering enables scalable, real-time social media monitoring.

Abstract

Automated narrative intelligence systems for social media monitoring face significant scalability challenges when relying on batch clustering methods to process continuous data streams. We investigate replacing offline HDBSCAN with online density-based clustering algorithms in a production narrative report generation pipeline that processes large volumes of multilingual social media data. While HDBSCAN effectively discovers hierarchical clusters and handles noise, its batch-only nature requires full retraining for each time window, limiting scalability and real-time adaptability. We evaluate online clustering methods with respect to cluster quality, computational efficiency, memory footprint, and integration with downstream narrative extraction. Our evaluation combines standard clustering metrics, narrative-specific measures, and human validation of cluster correctness to assess both structural quality and semantic interpretability. Experiments using sliding-window simulations on historical data from the Ukrainian information space reveal trade-offs between temporal stability and narrative coherence, with DenStream achieving the strongest overall performance. These findings bridge the gap between batch-oriented clustering approaches and the streaming requirements of large-scale narrative monitoring systems.