A Scalable Generative Graph Model with Community Structure

TL;DR

This paper introduces an improved, scalable generative graph model called BTER that accurately reproduces real-world network properties like degree distribution and community structure, suitable for large-scale graph benchmarking.

Contribution

The paper presents a scalable, parallelizable implementation of the BTER model that better fits real network data and can be tuned for specific degree and clustering profiles.

Findings

BTER outperforms other models in fitting real network data.

The implementation scales to billions of edges using Hadoop MapReduce.

BTER can be tuned for specific network properties.

Abstract

Network data is ubiquitous and growing, yet we lack realistic generative network models that can be calibrated to match real-world data. The recently proposed Block Two-Level Erdss-Renyi (BTER) model can be tuned to capture two fundamental properties: degree distribution and clustering coefficients. The latter is particularly important for reproducing graphs with community structure, such as social networks. In this paper, we compare BTER to other scalable models and show that it gives a better fit to real data. We provide a scalable implementation that requires only O(d_max) storage where d_max is the maximum number of neighbors for a single node. The generator is trivially parallelizable, and we show results for a Hadoop MapReduce implementation for a modeling a real-world web graph with over 4.6 billion edges. We propose that the BTER model can be used as a graph generator for benchmarking purposes and provide idealized degree distributions and clustering coefficient profiles that can be tuned for user specifications.