Sequential Changepoint Approach for Online Community Detection

TL;DR

This paper introduces new online algorithms for detecting community emergence in large networks modeled as Erdos-Renyi graphs, balancing detection speed and computational complexity.

Contribution

It develops three statistical changepoint detection algorithms—Exhaustive Search, mixture, and Hierarchical Mixture—for online community detection, with analysis and practical efficiency improvements.

Findings

The ES method has the best detection performance but is exponentially complex.

The mixture method offers polynomial complexity and effective detection.

The H-Mix method improves detection accuracy by addressing false alarms with dendrogram decomposition.

Abstract

We present new algorithms for detecting the emergence of a community in large networks from sequential observations. The networks are modeled using Erdos-Renyi random graphs with edges forming between nodes in the community with higher probability. Based on statistical changepoint detection methodology, we develop three algorithms: the Exhaustive Search (ES), the mixture, and the Hierarchical Mixture (H-Mix) methods. Performance of these methods is evaluated by the average run length (ARL), which captures the frequency of false alarms, and the detection delay. Numerical comparisons show that the ES method performs the best; however, it is exponentially complex. The mixture method is polynomially complex by exploiting the fact that the size of the community is typically small in a large network. However, it may react to a group of active edges that do not form a community. This issue is resolved by the H-Mix method, which is based on a dendrogram decomposition of the network. We present an asymptotic analytical expression for ARL of the mixture method when the threshold is large. Numerical simulation verifies that our approximation is accurate even in the non-asymptotic regime. Hence, it can be used to determine a desired threshold efficiently. Finally, numerical examples show that the mixture and the H-Mix methods can both detect a community quickly with a lower complexity than the ES method.