A method to evaluate the reliability of social media data for social network analysis

TL;DR

This paper presents a methodology to evaluate the reliability of social media data, specifically Twitter datasets, for social network analysis, highlighting how data collection methods impact analysis outcomes and providing guidelines for researchers.

Contribution

It introduces a systematic comparison methodology for assessing Twitter data reliability and demonstrates its importance through a case study showing data collection variations affect social network analysis.

Findings

Different data collection tools yield significantly different datasets.

Variations in datasets substantially influence social network analysis results.

Guidelines are proposed for reliable social media data collection.

Abstract

To study the effects of Online Social Network (OSN) activity on real-world offline events, researchers need access to OSN data, the reliability of which has particular implications for social network analysis. This relates not only to the completeness of any collected dataset, but also to constructing meaningful social and information networks from them. In this multidisciplinary study, we consider the question of constructing traditional social networks from OSN data and then present a measurement case study showing how the reliability of OSN data affects social network analyses. To this end we developed a systematic comparison methodology, which we applied to two parallel datasets we collected from Twitter. We found considerable differences in datasets collected with different tools and that these variations significantly alter the results of subsequent analyses. Our results lead to a set of guidelines for researchers planning to collect online data streams to infer social networks.