Efficient Detection of Communities with Significant Overlaps in Networks: Partial Community Merger Algorithm

TL;DR

This paper introduces PCMA, an efficient bottom-up algorithm for detecting overlapping communities in large networks, outperforming existing methods and revealing complex community structures in social networks.

Contribution

The paper presents a novel Partial Community Merger Algorithm that effectively detects overlapping communities with linear complexity, addressing noise and merger challenges.

Findings

PCMA outperforms four existing algorithms in accuracy and efficiency.

Millions of communities detected in large social networks show higher quality than metadata groups.

Significant overlaps in communities highlight the need for algorithms like PCMA.

Abstract

Detecting communities in large-scale networks is a challenging task when each vertex may belong to multiple communities, as is often the case in social networks. The multiple memberships of vertices and thus the strong overlaps among communities render many detection algorithms invalid. We develop a Partial Community Merger Algorithm (PCMA) for detecting communities with significant overlaps as well as slightly overlapping and disjoint ones. It is a bottom-up approach based on properly reassembling partial information of communities revealed in ego networks of vertices to reconstruct complete communities. Noise control and merger order are the two key issues in implementing this idea. We propose a novel similarity measure between two merged communities that can suppress noise and an efficient algorithm that recursively merges the most similar pair of communities. The validity and accuracy of PCMA is tested against two benchmarks and compared to four existing algorithms. It is the most efficient one with linear complexity and it outperforms the compared algorithms when vertices have multiple memberships. PCMA is applied to two huge online social networks, Friendster and Sina Weibo. Millions of communities are detected and they are of higher qualities than the corresponding metadata groups. We find that the latter should not be regarded as the ground-truth of structural communities. The significant overlapping pattern found in the detected communities confirms the need of new algorithms, such as PCMA, to handle multiple memberships of vertices in social networks.