Infinite Edge Partition Models for Overlapping Community Detection and Link Prediction

TL;DR

This paper introduces a hierarchical gamma process model for overlapping community detection and link prediction in large networks, automatically inferring community numbers and demonstrating scalability and high accuracy.

Contribution

It proposes a novel nonparametric Bayesian model that captures both homophily and stochastic equivalence, with efficient inference and connections to existing models.

Findings

Achieves state-of-the-art performance on real network data

Scales efficiently to large sparse networks

Automatically infers the number of communities

Abstract

A hierarchical gamma process infinite edge partition model is proposed to factorize the binary adjacency matrix of an unweighted undirected relational network under a Bernoulli-Poisson link. The model describes both homophily and stochastic equivalence, and is scalable to big sparse networks by focusing its computation on pairs of linked nodes. It can not only discover overlapping communities and inter-community interactions, but also predict missing edges. A simplified version omitting inter-community interactions is also provided and we reveal its interesting connections to existing models. The number of communities is automatically inferred in a nonparametric Bayesian manner, and efficient inference via Gibbs sampling is derived using novel data augmentation techniques. Experimental results on four real networks demonstrate the models' scalability and state-of-the-art performance.