Community detection for binary graphical models in high dimension

TL;DR

This paper introduces simple spectral and aggregated methods for community detection in high-dimensional binary graphical models, achieving near-optimal misclassification rates and exact recovery under certain conditions.

Contribution

It proposes two novel methods that do not require prior knowledge of model parameters, with proven theoretical guarantees for community detection in high-dimensional settings.

Findings

Misclassification rates vanish when observation time T is much larger than number of components N.

Exact recovery is achievable with high probability when T is much larger than N squared.

Methods are near-optimal and do not need prior parameter knowledge.

Abstract

Let $N$ components be partitioned into two communities, denoted ${\cal P}_+$ and ${\cal P}_-$, possibly of different sizes. Assume that they are connected via a directed and weighted Erd\"os-R\'enyi (DWER) random graph with unknown parameter $ p \in (0, 1).$ The weights assigned to the existing connections are of mean-field-type, scaling as $N^{-1}$. At each time \modif{step}, we observe the state of each component: either it sends some signal to its successors (in the directed graph) or remains silent otherwise. In this paper, we show that it is possible to find the communities ${\cal P}_+$ and ${\cal P}_-$ based only on the activity of the $N$ components observed over $T$ time units. More specifically, we propose \modif{ two simple methods, an aggregated method and a spectral method, whose {\it misclassification rates} vanish as long as $T \gg N$ (up to log terms). This condition is proved to be near-optimal in the minimax sense. Moreover, under the stronger condition $T \gg N^2$ (up to log terms), the aggregated method is shown to achieve {\it exact recovery} with probability tending to $1$. } Interestingly, these simple \modif{methods} do not require any prior knowledge of the other model parameters (e.g. the edge probability $p$). The key step in our analysis is to derive an asymptotic approximation of the 1-lagged covariance matrix associated to the states of the $N$ components, as $N$ diverges. This asymptotic approximation relies on the study of the behavior of the solutions of a \modif{Stein-type} matrix equation satisfied by the simultaneous (0-lagged) covariance matrix associated to the states of the components. This study is challenging, especially because the simultaneous covariance matrix is random since it depends on the underlying DWER random graph.