FastEnsemble: scalable ensemble clustering on large networks

TL;DR

FastEnsemble is a scalable consensus clustering method that improves accuracy and speed for large networks, effectively reducing variability in community detection results across different algorithms and parameters.

Contribution

Introduces FastEnsemble, a novel, scalable consensus clustering approach that outperforms existing methods in accuracy and efficiency on large networks.

Findings

FastEnsemble yields more accurate community detection than ECG and FastConsensus.

It is capable of handling networks with over 3 million nodes efficiently.

Consensus clustering helps mitigate resolution limit issues in network analysis.

Abstract

Many community detection algorithms are inherently stochastic, leading to variations in their output depending on input parameters and random seeds. This variability makes the results of a single run of these algorithms less reliable. Moreover, different clustering algorithms, optimization criteria (e.g., modularity, the Constant Potts model), and resolution values can result in substantially different partitions on the same network. Consensus clustering methods, such as ECG and FastConsensus, have been proposed to reduce the instability of non-deterministic algorithms and improve their accuracy by combining a set of partitions resulting from multiple runs of a clustering algorithm. In this work, we introduce FastEnsemble, a new consensus clustering method. Our results on a wide range of synthetic networks show that FastEnsemble produces more accurate clusterings than two other consensus clustering methods, ECG and FastConsensus, for many model conditions. Furthermore, FastEnsemble is fast enough to be used on networks with more than 3 million nodes, and so improves on the speed and scalability of FastConsensus. Finally, we showcase the utility of consensus clustering methods in mitigating the effect of resolution limit and clustering networks that are only partially covered by communities.