MOStream: A Modular and Self-Optimizing Data Stream Clustering Algorithm

TL;DR

MOStream is a novel modular data stream clustering algorithm that dynamically balances accuracy and efficiency, adapting to diverse data characteristics and outperforming existing methods in real-world and synthetic benchmarks.

Contribution

It introduces a self-optimizing, modular framework with four key design dimensions, enabling flexible adaptation and improved clustering performance in data streams.

Findings

MOStream outperforms 9 benchmark algorithms in accuracy and throughput.

It demonstrates high adaptability across diverse datasets.

The modular design enhances flexibility and self-optimization.

Abstract

Data stream clustering is a critical operation in various real-world applications, ranging from the Internet of Things (IoT) to social media and financial systems. Existing data stream clustering algorithms, while effective to varying extents, often lack the flexibility and self-optimization capabilities needed to adapt to diverse workload characteristics such as outlier, cluster evolution and changing dimensions in data points. These limitations manifest in suboptimal clustering accuracy and computational inefficiency. In this paper, we introduce MOStream, a modular and self-optimizing data stream clustering algorithm designed to dynamically balance clustering accuracy and computational efficiency at runtime. MOStream distinguishes itself by its adaptivity, clearly demarcating four pivotal design dimensions: the summarizing data structure, the window model for handling data temporality, the outlier detection mechanism, and the refinement strategy for improving cluster quality. This clear separation facilitates flexible adaptation to varying design choices and enhances its adaptability to a wide array of application contexts. We conduct a rigorous performance evaluation of MOStream, employing diverse configurations and benchmarking it against 9 representative data stream clustering algorithms on 4 real-world datasets and 3 synthetic datasets. Our empirical results demonstrate that MOStream consistently surpasses competing algorithms in terms of clustering accuracy, processing throughput, and adaptability to varying data stream characteristics.