Stochastic fluctuations and the detectability limit of network communities

TL;DR

This paper investigates the fundamental limits of detecting communities in networks, revealing that stochastic fluctuations cause the true community structure to be lost earlier than previously thought, and provides an analytical estimate of this detectability threshold.

Contribution

The study introduces an analytical scheme based on branching processes to accurately estimate the true detectability limit considering stochastic fluctuations.

Findings

Community detection fails before the in-degree/out-degree ratio matches a random network.

All tested modularity-based algorithms exhibit similar detectability thresholds.

Stochastic fluctuations are crucial for defining the true limits of community detectability.

Abstract

We have analyzed the detectability limits of network communities in the framework of the popular Girvan and Newman benchmark. By carefully taking into account the inevitable stochastic fluctuations that affect the construction of each and every instance of the benchmark, we come to the conclusions that the native, putative partition of the network is completely lost even before the in-degree/out-degree ratio becomes equal to the one of a structure-less Erd\"os-R\'enyi network. We develop a simple iterative scheme, analytically well described by an infinite branching-process, to provide an estimate of the true detectability limit. Using various algorithms based on modularity optimization, we show that all of them behave (semi-quantitatively) in the same way, with the same functional form of the detectability threshold as a function of the network parameters. Because the same behavior has also been found by further modularity-optimization methods and for methods based on different heuristics implementations, we conclude that indeed a correct definition of the detectability limit must take into account the stochastic fluctuations of the network construction.