Enhancing Stability and Assessing Uncertainty in Community Detection through a Consensus-based Approach

TL;DR

This paper introduces a consensus-based framework called CCD to improve stability and uncertainty assessment in community detection, addressing non-determinism and outlier detection in network analysis.

Contribution

The paper presents a novel, versatile framework that quantifies uncertainty and manages outliers in community detection across various algorithms.

Findings

Effective uncertainty quantification for networks and nodes

Enhanced stability of community detection results

Three strategies for outlier handling

Abstract

Complex data in social and natural sciences find effective representation through networks, wherein quantitative and categorical information can be associated with nodes and connecting edges. The internal structure of networks can be explored using unsupervised machine learning methods known as community detection algorithms. The process of community detection is inherently subject to uncertainty as algorithms utilize heuristic approaches and randomised procedures to explore vast solution spaces, resulting in non-deterministic outcomes and variability in detected communities across multiple runs. Moreover, many algorithms are not designed to identify outliers and may fail to take into account that a network is an unordered mathematical entity. The main aim of our work is to address these issues through a consensus-based approach by introducing a new framework called Consensus Community Detection (CCD). Our method can be applied to different community detection algorithms, allowing the quantification of uncertainty for the whole network as well as for each node, and providing three strategies for dealing with outliers: incorporate, highlight, or group. The effectiveness of our approach is evaluated on artificial benchmark networks.