A shadowing problem in the detection of overlapping communities: lifting the resolution limit through a cascading procedure

TL;DR

This paper identifies a shadowing problem in community detection algorithms where larger communities overshadow smaller ones, and proposes a cascading method to improve multi-scale community detection in networks.

Contribution

It introduces a generic cascading approach that lifts the resolution limit caused by shadowing, enabling detection of overlooked communities across scales.

Findings

Cascading procedure improves detection of small communities

Shadowing accounts for most undetected communities

Method effective on real and artificial networks

Abstract

Community detection is the process of assigning nodes and links in significant communities (e.g. clusters, function modules) and its development has led to a better understanding of complex networks. When applied to sizable networks, we argue that most detection algorithms correctly identify prominent communities, but fail to do so across multiple scales. As a result, a significant fraction of the network is left uncharted. We show that this problem stems from larger or denser communities overshadowing smaller or sparser ones, and that this effect accounts for most of the undetected communities and unassigned links. We propose a generic cascading approach to community detection that circumvents the problem. Using real and artificial network datasets with three widely used community detection algorithms, we show how a simple cascading procedure allows for the detection of the missing communities. This work highlights a new detection limit of community structure, and we hope that our approach can inspire better community detection algorithms.