CluStRE: Streaming Graph Clustering with Multi-Stage Refinement

TL;DR

CluStRE is a streaming graph clustering algorithm that achieves high-quality results with reduced memory and faster processing by using multi-stage refinement and dynamic quotient graph construction.

Contribution

It introduces a novel multi-stage refinement approach for streaming graph clustering that balances efficiency and quality, bridging the gap with in-memory methods.

Findings

Improves clustering solution quality by 89.8%

Operates 2.6 times faster than state-of-the-art streaming algorithms

Uses less than two-thirds of the memory of existing methods

Abstract

We present CluStRE, a novel streaming graph clustering algorithm that balances computational efficiency with high-quality clustering using multi-stage refinement. Unlike traditional in-memory clustering approaches, CluStRE processes graphs in a streaming setting, significantly reducing memory overhead while leveraging re-streaming and evolutionary heuristics to improve solution quality. Our method dynamically constructs a quotient graph, enabling modularity-based optimization while efficiently handling large-scale graphs. We introduce multiple configurations of CluStRE to provide trade-offs between speed, memory consumption, and clustering quality. Experimental evaluations demonstrate that CluStRE improves solution quality by 89.8%, operates 2.6 times faster, and uses less than two-thirds of the memory required by the state-of-the-art streaming clustering algorithm on average. Moreover, our strongest mode enhances solution quality by up to 150% on average. With this, CluStRE achieves comparable solution quality to in-memory algorithms, i.e. over 96% of the quality of clustering approaches, including Louvain, effectively bridging the gap between streaming and traditional clustering methods.