Community Detection with the Canonical Ensemble

TL;DR

This paper proposes a hypothesis testing approach to community detection in networks, using null models from the canonical ensemble to assess the significance of observed community structures.

Contribution

It introduces a novel statistical framework for community detection that provides definitive evidence for community structure, contrasting with traditional algorithmic methods.

Findings

The method effectively distinguishes significant communities in real and synthetic networks.

It offers a more definitive answer to community presence than generic algorithms.

Compared to Bayesian stochastic block models, it provides clearer hypothesis testing results.

Abstract

Network community detection is usually considered as an unsupervised learning problem. Given a network, the aim is to partition it using some general purpose algorithm. In this paper we instead treat community detection as a hypothesis testing problem. Given a network, we examine the evidence for specific community structure in the observed network compared to a null model. To do this we define an appropriate test statistic, analogous to a z-score, and several null models derived from maximising entropy under different constraints in the canonical ensemble. We demonstrate the application of this method on real and synthetic data and contrast our method to Bayesian approaches based on the stochastic block model. We demonstrate that this method gives definitive answers to concrete questions, which can be more useful to analysts than the output of a generic algorithm.