Distributed Pseudo-Likelihood Method for Community Detection in Large-Scale Networks

TL;DR

This paper introduces a distributed pseudo-likelihood method (DPL) for efficient community detection in large-scale networks, reducing computational complexity and extending to degree-corrected models, with strong theoretical and empirical validation.

Contribution

The paper presents a novel distributed algorithm for community detection that scales to large networks, improves computational efficiency, and extends to degree-corrected stochastic block models.

Findings

DPL significantly reduces computational complexity.

The method achieves accurate community detection in large networks.

Theoretical analysis confirms statistical consistency.

Abstract

This paper proposes a distributed pseudo-likelihood method (DPL) to conveniently identify the community structure of large-scale networks. Specifically, we first propose a block-wise splitting method to divide large-scale network data into several subnetworks and distribute them among multiple workers. For simplicity, we assume the classical stochastic block model. Then, the DPL algorithm is iteratively implemented for the distributed optimization of the sum of the local pseudo-likelihood functions. At each iteration, the worker updates its local community labels and communicates with the master. The master then broadcasts the combined estimator to each worker for the new iterative steps. Based on the distributed system, DPL significantly reduces the computational complexity of the traditional pseudo-likelihood method using a single machine. Furthermore, to ensure statistical accuracy, we theoretically discuss the requirements of the worker sample size. Moreover, we extend the DPL method to estimate degree-corrected stochastic block models. The superior performance of the proposed distributed algorithm is demonstrated through extensive numerical studies and real data analysis.