Generalized Measures for the Evaluation of Community Detection Methods

TL;DR

This paper reviews traditional community detection evaluation measures, identifies their limitations regarding network structure, proposes modifications to improve relevance, and demonstrates that the modified NMI performs best on synthetic networks.

Contribution

It introduces modified versions of purity, Rand index, and NMI that incorporate network structure, enhancing evaluation accuracy for community detection methods.

Findings

Modified NMI outperforms traditional measures on synthetic data

Traditional measures ignore network topology, leading to potential misinterpretations

Modified measures show higher relevance to community structure properties

Abstract

Community detection can be considered as a variant of cluster analysis applied to complex networks. For this reason, all existing studies have been using tools derived from this field when evaluating community detection algorithms. However, those are not completely relevant in the context of network analysis, because they ignore an essential part of the available information: the network structure. Therefore, they can lead to incorrect interpretations. In this article, we review these measures, and illustrate this limitation. We propose a modification to solve this problem, and apply it to the three most widespread measures: purity, Rand index and normalized mutual information (NMI). We then perform an experimental evaluation on artificially generated networks with realistic community structure. We assess the relevance of the modified measures by comparison with their traditional counterparts, and also relatively to the topological properties of the community structures. On these data, the modified NMI turns out to provide the most relevant results.