Parallel Algorithms for Median Consensus Clustering in Complex Networks

TL;DR

This paper introduces a parallel algorithm for median consensus clustering in complex networks, effectively capturing community structures with high speed and accuracy, especially suitable for large-scale graphs.

Contribution

It presents a novel parallel greedy algorithm that incorporates graph structure into median consensus clustering, significantly improving speed and accuracy over existing methods.

Findings

Achieves 35x speedup with 64 cores on large graphs

More accurately captures community structures in graphs

Outperforms existing median clustering approaches in quality and efficiency

Abstract

We develop an algorithm that finds the consensus of many different clustering solutions of a graph. We formulate the problem as a median set partitioning problem and propose a greedy optimization technique. Unlike other approaches that find median set partitions, our algorithm takes graph structure into account and finds a comparable quality solution much faster than the other approaches. For graphs with known communities, our consensus partition captures the actual community structure more accurately than alternative approaches. To make it applicable to large graphs, we remove sequential dependencies from our algorithm and design a parallel algorithm. Our parallel algorithm achieves 35x speedup when utilizing 64 processing cores for large real-world graphs from single-cell experiments.