Resampling effects on significance analysis of network clustering and ranking

TL;DR

This paper investigates how different resampling methods affect the statistical significance analysis of network communities, especially in scientific citation networks, highlighting the importance of dependency preservation for accurate results.

Contribution

It compares parametric citation resampling with non-parametric article resampling, revealing how dependency assumptions influence significance analysis outcomes.

Findings

Citation resampling underestimates link weight variance.

Maintaining dependencies in resampling improves significance analysis.

A simple parametric scheme can approximate article resampling variances.

Abstract

Community detection helps us simplify the complex configuration of networks, but communities are reliable only if they are statistically significant. To detect statistically significant communities, a common approach is to resample the original network and analyze the communities. But resampling assumes independence between samples, while the components of a network are inherently dependent. Therefore, we must understand how breaking dependencies between resampled components affects the results of the significance analysis. Here we use scientific communication as a model system to analyze this effect. Our dataset includes citations among articles published in journals in the years 1984-2010. We compare parametric resampling of citations with non-parametric article resampling. While citation resampling breaks link dependencies, article resampling maintains such dependencies. We find that citation resampling underestimates the variance of link weights. Moreover, this underestimation explains most of the differences in the significance analysis of ranking and clustering. Therefore, when only link weights are available and article resampling is not an option, we suggest a simple parametric resampling scheme that generates link-weight variances close to the link-weight variances of article resampling. Nevertheless, when we highlight and summarize important structural changes in science, the more dependencies we can maintain in the resampling scheme, the earlier we can predict structural change.