Modelling the Spread of New Information on X

TL;DR

This paper investigates predicting the reposting of new, unseen information on X (formerly Twitter), highlighting the importance of user profile features over content features for out-of-distribution prediction accuracy.

Contribution

It demonstrates that incorporating user profile and behavior features significantly improves out-of-distribution repost prediction, outperforming content-based models.

Findings

Content features perform well in in-distribution scenarios.

Out-of-distribution prediction accuracy drops with content features alone.

User profile features greatly enhance out-of-distribution prediction, increasing F1 score from 0.117 to 0.705.

Abstract

There has been considerable interest in modelling the spread of information on X (formerly Twitter) using machine learning models. Here, we consider the problem of predicting the reposting of new information, i.e., when a user propagates information about a topic previously unseen by the user. In existing work, information and users are randomly assigned to a test or training set, ensuring that both sets are drawn from the same distribution. In the spread of new information, the problem becomes an out-of-distribution classification task. Our experimental results reveal that while existing algorithms, which predominantly use features derived from the content of posts, perform well when the training and test distributions are the same, they perform much worse when the test set is out-of-distribution, i.e., when the topic of the testing data is absent from the training data. We then show that if the post features are supplemented or replaced with features derived from user profiles and past behaviours, the out-of-distribution prediction is greatly improved, with the F1 score increasing from 0.117 to 0.705. Our experimental results suggest that a significant component of reposting behaviour for previously unseen topics can be predicted from user profiles and past behaviours, and is largely content-agnostic.