Computing the statistical significance of optimized communities in networks

TL;DR

This paper introduces FOCS, a scalable and stable algorithm for assessing the statistical significance of communities in various types of networks, improving reliability over existing methods.

Contribution

It generalizes null models to bipartite graphs and presents FOCS, a novel, scalable significance scoring algorithm that enhances stability and detection accuracy.

Findings

FOCS is highly scalable and applicable to different graph types.

It offers improved numerical stability over existing methods.

FOCS better balances detection power and false positive rates.

Abstract

It is often of interest to find communities in network data for unsupervised learning, feature discovery, anomaly detection, or scientific study. The vast majority of community detection methods proceed via optimization of a quality function, which is possible even on random networks without communities. Therefore there is usually not an easy way to tell if a community is "significant", in this context meaning more internally connected than would be expected under a random graph model without communities. This paper generalizes existing null models for this purpose to bipartite graphs, and introduces a new significance scoring algorithm called Fast Optimized Community Significance (FOCS) that is highly scalable and agnostic to the type of graph. Furthermore, compared with existing methods on unipartite graphs, FOCS is more numerically stable and better balances the trade-off between detection power and false positives.