Scalable Privacy-Compliant Virality Prediction on Twitter

TL;DR

This paper introduces a scalable, privacy-compliant framework for predicting Twitter content virality using a novel gradient boosting method that achieves state-of-the-art ranking performance across multiple languages.

Contribution

It presents a new data acquisition and analysis framework that ensures privacy, along with a gradient boosting model that provides explainable, accurate virality predictions on large-scale Twitter data.

Findings

Achieved state-of-the-art virality ranking results

Model performs well across 18 languages

Framework ensures privacy compliance during data processing

Abstract

The digital town hall of Twitter becomes a preferred medium of communication for individuals and organizations across the globe. Some of them reach audiences of millions, while others struggle to get noticed. Given the impact of social media, the question remains more relevant than ever: how to model the dynamics of attention in Twitter. Researchers around the world turn to machine learning to predict the most influential tweets and authors, navigating the volume, velocity, and variety of social big data, with many compromises. In this paper, we revisit content popularity prediction on Twitter. We argue that strict alignment of data acquisition, storage and analysis algorithms is necessary to avoid the common trade-offs between scalability, accuracy and privacy compliance. We propose a new framework for the rapid acquisition of large-scale datasets, high accuracy supervisory signal and multilanguage sentiment prediction while respecting every privacy request applicable. We then apply a novel gradient boosting framework to achieve state-of-the-art results in virality ranking, already before including tweet's visual or propagation features. Our Gradient Boosted Regression Tree is the first to offer explainable, strong ranking performance on benchmark datasets. Since the analysis focused on features available early, the model is immediately applicable to incoming tweets in 18 languages.