Community Detection in High-Dimensional Graph Ensembles

TL;DR

This paper develops a new spectral method for community detection in high-dimensional degree-corrected stochastic block models, overcoming heterogeneity issues and providing theoretical and empirical validation.

Contribution

It introduces a matrix transformation that handles degree heterogeneity and proposes a spectral test with significance control and conjectured power properties.

Findings

The transformed matrix preserves community structure.

The proposed test controls significance levels effectively.

Empirical results support the test's asymptotic power.

Abstract

Detecting communities in high-dimensional graphs can be achieved by applying random matrix theory where the adjacency matrix of the graph is modeled by a Stochastic Block Model (SBM). However, the SBM makes an unrealistic assumption that the edge probabilities are homogeneous within communities, i.e., the edges occur with the same probabilities. The Degree-Corrected SBM is a generalization of the SBM that allows these edge probabilities to be different, but existing results from random matrix theory are not directly applicable to this heterogeneous model. In this paper, we derive a transformation of the adjacency matrix that eliminates this heterogeneity and preserves the relevant eigenstructure for community detection. We propose a test based on the extreme eigenvalues of this transformed matrix and (1) provide a method for controlling the significance level, (2) formulate a conjecture that the test achieves power one for all positive significance levels in the limit as the number of nodes approaches infinity, and (3) provide empirical evidence and theory supporting these claims.