Designing Parallel Algorithms for Community Detection using Arachne

TL;DR

This paper introduces parallel algorithms for community detection using Arachne, achieving significant speedups over existing tools and demonstrating scalable performance for large-scale graph analysis.

Contribution

It presents the first parallel implementations of Label Propagation and Louvain algorithms in Arachne, optimized for large-scale graph community detection.

Findings

Speedups of up to 710x over NetworkX

Achieved 75x speedup over igraph

Demonstrated scalability with varying thread counts

Abstract

The rise of graph data in various fields calls for efficient and scalable community detection algorithms. In this paper, we present parallel implementations of two widely used algorithms: Label Propagation and Louvain, specifically designed to leverage the capabilities of Arachne, which is a Python-accessible open-source framework for large-scale graph analysis. Our implementations achieve substantial speedups over existing Python-based tools like NetworkX and igraph, which lack efficient parallelization, and are competitive with parallel frameworks such as NetworKit. Experimental results show that Arachne-based methods outperform these baselines, achieving speedups of up to 710x over NetworkX, 75x over igraph, and 12x over NetworKit. Additionally, we analyze the scalability of our implementation under varying thread counts, demonstrating how different phases contribute to overall performance gains of the parallel Louvain algorithm. Arachne, including our community detection implementation, is open-source and available at https://github.com/Bears-R-Us/arkouda-njit .