Tail-scope: Using friends to estimate heavy tails of degree distributions in large-scale complex networks

TL;DR

This paper introduces the tail-scope method, leveraging local network bias to efficiently estimate heavy-tailed degree distributions in large-scale networks, outperforming uniform sampling especially for high-degree nodes.

Contribution

The paper presents a novel tail-scope sampling technique based on the friendship paradox, improving heavy tail estimation in large networks using limited local data.

Findings

Tail-scope outperforms uniform sampling for high-degree nodes.

Hybrid method combines tail-scope and uniform sampling to cover all degree ranges.

Structural heterogeneities enable effective network structure estimation with limited data.

Abstract

Many complex networks in natural and social phenomena have often been characterized by heavy-tailed degree distributions. However, due to rapidly growing size of network data and concerns on privacy issues about using these data, it becomes more difficult to analyze complete data sets. Thus, it is crucial to devise effective and efficient estimation methods for heavy tails of degree distributions in large-scale networks only using local information of a small fraction of sampled nodes. Here we propose a tail-scope method based on local observational bias of the friendship paradox. We show that the tail-scope method outperforms the uniform node sampling for estimating heavy tails of degree distributions, while the opposite tendency is observed in the range of small degrees. In order to take advantages of both sampling methods, we devise the hybrid method that successfully recovers the whole range of degree distributions. Our tail-scope method shows how structural heterogeneities of large-scale complex networks can be used to effectively reveal the network structure only with limited local information.