Towards realistic artificial benchmark for community detection algorithms evaluation

TL;DR

This paper investigates how the realism of artificial benchmarks affects community detection algorithm performance, proposing enhanced models that better mimic real-world network properties and reveal decreased algorithm effectiveness.

Contribution

It introduces two new random models based on the LFR method that produce more realistic artificial networks for community detection benchmarking.

Findings

Enhanced models generate networks with properties closer to real-world networks.

Community detection algorithms perform worse on more realistic benchmarks.

Proposed models improve the validity of algorithm evaluation.

Abstract

Assessing the partitioning performance of community detection algorithms is one of the most important issues in complex network analysis. Artificially generated networks are often used as benchmarks for this purpose. However, previous studies showed their level of realism have a significant effect on the algorithms performance. In this study, we adopt a thorough experimental approach to tackle this problem and investigate this effect. To assess the level of realism, we use consensual network topological properties. Based on the LFR method, the most realistic generative method to date, we propose two alternative random models to replace the Configuration Model originally used in this algorithm, in order to increase its realism. Experimental results show both modifications allow generating collections of community-structured artificial networks whose topological properties are closer to those encountered in real-world networks. Moreover, the results obtained with eleven popular community identification algorithms on these benchmarks show their performance decrease on more realistic networks.