When random initializations help: a study of variational inference for community detection

TL;DR

This paper investigates how variational inference, specifically mean field approximation, performs in community detection within stochastic block models, highlighting the impact of initializations on convergence to true communities.

Contribution

It provides a theoretical analysis of the convergence behavior of variational inference in community detection, emphasizing the role of initialization and parameter estimation.

Findings

Proper initialization can lead to convergence to the true community structure.

Random initialization often results in convergence to uninformative local optima.

Parameter estimation affects the convergence behavior significantly.

Abstract

Variational approximation has been widely used in large-scale Bayesian inference recently, the simplest kind of which involves imposing a mean field assumption to approximate complicated latent structures. Despite the computational scalability of mean field, theoretical studies of its loss function surface and the convergence behavior of iterative updates for optimizing the loss are far from complete. In this paper, we focus on the problem of community detection for a simple two-class Stochastic Blockmodel (SBM) with equal class sizes. Using batch co-ordinate ascent (BCAVI) for updates, we show different convergence behavior with respect to different initializations. When the parameters are known or estimated within a reasonable range and held fixed, we characterize conditions under which an initialization can converge to the ground truth. On the other hand, when the parameters need to be estimated iteratively, a random initialization will converge to an uninformative local optimum.