Upscaling human activity data: an ecological perspective

TL;DR

This paper introduces a statistical framework based on ecological inference to predict global human activity features from local data, demonstrating robustness and accuracy across multiple datasets.

Contribution

It presents a novel ecological-inspired method for inferring large-scale human activity patterns from small samples, bridging biodiversity models with social data analysis.

Findings

Accurately predicts global features from local data with small errors.

Can forecast changes in popularity of hashtags or words across scales.

Applicable to diverse datasets like email, Twitter, Wikipedia, and books.

Abstract

In recent years we have witnessed an explosion of data collected for different human dynamics, from email communication to social networks activities. Extract useful information from these huge data sets represents a major challenge. In the last decades, statistical regularities has been widely observed in human activities and various models have been proposed. Here we move from modeling to inference and propose a statistical framework capable to predict global features of human activities from local knowledge. We consider four data sets of human activities: email communication, Twitter posts, Wikipedia articles and Gutenberg books. From the statistics of local activities, such as sent emails per senders, post per hashtags and word occurrences collected in a small sample of the considered dataset, we infer global features, as the number of senders, hashtags and words at the global scale. Our estimates are robust and accurate with a small relative error. Moreover, we predict how abundance of a hashtag or of a word may change through scales. Thus, observing a small portion of tweets and the popularity of a given hashtag among them, we can estimate whether it will remain popular or not in the unseen part of the network. Our approach is grounded on statistical ecology as we discover inference of unseen human activity hallmarks can be mapped into the unseen species problem in biodiversity. Our findings may have applications to different areas, from resource management in emails to collective attention monitoring in Twitter and to language learning process in word databases.