Towards real-time community detection in large networks

TL;DR

This paper enhances a linear-time community detection algorithm for large social networks, making it faster and more accurate for real-time analysis of massive graph data.

Contribution

It extends a linear-time label propagation algorithm with heuristics, improving its reliability and accuracy for real-time community detection in large-scale networks.

Findings

The extended algorithm is faster than the original.

It achieves better accuracy than modularity-based methods.

It successfully processes networks with over 58 million edges.

Abstract

The recent boom of large-scale Online Social Networks (OSNs) both enables and necessitates the use of parallelisable and scalable computational techniques for their analysis. We examine the problem of real-time community detection and a recently proposed linear time - O(m) on a network with m edges - label propagation or "epidemic" community detection algorithm. We identify characteristics and drawbacks of the algorithm and extend it by incorporating different heuristics to facilitate reliable and multifunctional real-time community detection. With limited computational resources, we employ the algorithm on OSN data with 1 million nodes and about 58 million directed edges. Experiments and benchmarks reveal that the extended algorithm is not only faster but its community detection accuracy is compared favourably over popular modularity-gain optimization algorithms known to suffer from their resolution limits.