The Value of Using Big Data Technologies in Computational Social Science

TL;DR

This paper demonstrates how scalable open source Big Data technologies can be effectively used to acquire, process, and analyze massive social media datasets, revealing social information landscapes within Twitter for social science research.

Contribution

It introduces a process for managing social media data with Big Data tools, showcasing their feasibility and value in social science investigations.

Findings

Successful integration of open source Big Data technologies for social media data collection.

Discovery of social information landscapes within Twitter datasets.

Validation of scalable technologies for complex social data analysis.

Abstract

The discovery of phenomena in social networks has prompted renewed interests in the field. Data in social networks however can be massive, requiring scalable Big Data architecture. Conversely, research in Big Data needs the volume and velocity of social media data for testing its scalability. Not only so, appropriate data processing and mining of acquired datasets involve complex issues in the variety, veracity, and variability of the data, after which visualisation must occur before we can see fruition in our efforts. This article presents topical, multimodal, and longitudinal social media datasets from the integration of various scalable open source technologies. The article details the process that led to the discovery of social information landscapes within the Twitter social network, highlighting the experience of dealing with social media datasets, using a funneling approach so that data becomes manageable. The article demonstrated the feasibility and value of using scalable open source technologies for acquiring massive, connected datasets for research in the social sciences.