Revealing Multiple Layers of Hidden Community Structure in Networks

TL;DR

This paper introduces HICODE, a novel algorithm that uncovers hidden, weaker community structures in networks by iteratively removing dominant communities, revealing meaningful overlapping communities often missed by existing methods.

Contribution

HICODE provides a new framework for detecting both strong and hidden communities in networks, improving the identification of meaningful overlapping structures beyond current algorithms.

Findings

HICODE uncovers hidden communities in real and synthetic networks.

It outperforms baseline algorithms in detecting weaker community structures.

Applied to a student network, it revealed communities with high accuracy.

Abstract

We introduce a new conception of community structure, which we refer to as hidden community structure. Hidden community structure refers to a specific type of overlapping community structure, in which the detection of weak, but meaningful, communities is hindered by the presence of stronger communities. We present Hidden Community Detection HICODE, an algorithm template that identifies both the strong, dominant community structure as well as the weaker, hidden community structure in networks. HICODE begins by first applying an existing community detection algorithm to a network, and then removing the structure of the detected communities from the network. In this way, the structure of the weaker communities becomes visible. Through application of HICODE, we demonstrate that a wide variety of real networks from different domains contain many communities that, though meaningful, are not detected by any of the popular community detection algorithms that we consider. Additionally, on both real and synthetic networks containing a hidden ground-truth community structure, HICODE uncovers this structure better than any baseline algorithms that we compared against. For example, on a real network of undergraduate students that can be partitioned either by `Dorm' (residence hall) or `Year', we see that HICODE uncovers the weaker `Year' communities with a JCRecall score (a recall-based metric that we define in the text) of over 0.7, while the baseline algorithms achieve scores below 0.2.