Large Scale Community-Aware Network Generation

TL;DR

This paper introduces RECCS+ and RECCS++, scalable algorithms for synthetic network generation that preserve community structures, achieving significant speedups and enabling the handling of extremely large networks.

Contribution

The paper presents enhanced, parallelized versions of RECCS that significantly improve speed and scalability for large-scale community-aware network generation.

Findings

RECCS+ achieves up to 49x speedup over the original.

RECCS++ achieves up to 139x speedup with a slight accuracy tradeoff.

RECCS++ can scale to networks with over 100 million nodes.

Abstract

Community detection, or network clustering, is used to identify latent community structure in networks. Due to the scarcity of labeled ground truth in real-world networks, evaluating these algorithms poses significant challenges. To address this, researchers use synthetic network generators that produce networks with ground-truth community labels. RECCS is one such algorithm that takes a network and its clustering as input and generates a synthetic network through a modular pipeline. Each generated ground truth cluster preserves key characteristics of the corresponding input cluster, including connectivity, minimum degree, and degree sequence distribution. The output consists of a synthetically generated network, and disjoint ground truth cluster labels for all nodes. In this paper, we present two enhanced versions: RECCS+ and RECCS++. RECCS+ maintains algorithmic fidelity to the original RECCS while introducing parallelization through an orchestrator that coordinates algorithmic components across multiple processes and employs multithreading. RECCS++ builds upon this foundation with additional algorithmic optimizations to achieve further speedup. Our experimental results demonstrate that RECCS+ and RECCS++ achieve speedups of up to 49x and 139x respectively on our benchmark datasets, with RECCS++'s additional performance gains involving a modest accuracy tradeoff. With this newfound performance, RECCS++ can now scale to networks with over 100 million nodes and nearly 2 billion edges.