Still out there: Modeling and Identifying Russian Troll Accounts on Twitter

TL;DR

This paper develops machine learning models to identify Russian troll accounts on Twitter, demonstrating that such accounts are still active today and can be distinguished from genuine users with high accuracy using behavioral and linguistic features.

Contribution

The study introduces a predictive model for Russian troll detection on Twitter and applies it to current data, revealing ongoing troll activity and complex identity management.

Findings

Achieved 78.5% precision and 98.9% AUC in distinguishing trolls from non-trolls.

Found up to 2.6% of top journalists' mentions are Russian trolls.

Confirmed trolls are actively managing their online identities, not just automated bots.

Abstract

There is evidence that Russia's Internet Research Agency attempted to interfere with the 2016 U.S. election by running fake accounts on Twitter - often referred to as "Russian trolls". In this work, we: 1) develop machine learning models that predict whether a Twitter account is a Russian troll within a set of 170K control accounts; and, 2) demonstrate that it is possible to use this model to find active accounts on Twitter still likely acting on behalf of the Russian state. Using both behavioral and linguistic features, we show that it is possible to distinguish between a troll and a non-troll with a precision of 78.5% and an AUC of 98.9%, under cross-validation. Applying the model to out-of-sample accounts still active today, we find that up to 2.6% of top journalists' mentions are occupied by Russian trolls. These findings imply that the Russian trolls are very likely still active today. Additional analysis shows that they are not merely software-controlled bots, and manage their online identities in various complex ways. Finally, we argue that if it is possible to discover these accounts using externally - accessible data, then the platforms - with access to a variety of private internal signals - should succeed at similar or better rates.