Toxicity in State Sponsored Information Operations

TL;DR

This paper provides a comprehensive analysis of toxic language in state-sponsored information operations on social media, revealing strategic deployment patterns and engagement disparities across geopolitical entities.

Contribution

First systematic study quantifying toxic content in state-sponsored campaigns using large-scale social media data and API-based detection methods.

Findings

Toxic content accounts for 1.53% of posts but has high engagement.

Russian influence operations' toxic content receives significantly higher engagement.

Toxic content is strategically deployed in specific geopolitical contexts.

Abstract

State-sponsored information operations (IOs) increasingly influence global discourse on social media platforms, yet their emotional and rhetorical strategies remain inadequately characterized in scientific literature. This study presents the first comprehensive analysis of toxic language deployment within such campaigns, examining 56 million posts from over 42 thousand accounts linked to 18 distinct geopolitical entities on X/Twitter. Using Google's Perspective API, we systematically detect and quantify six categories of toxic content and analyze their distribution across national origins, linguistic structures, and engagement metrics, providing essential information regarding the underlying patterns of such operations. Our findings reveal that while toxic content constitutes only 1.53% of all posts, they are associated with disproportionately high engagement and appear to be strategically deployed in specific geopolitical contexts. Notably, toxic content originating from Russian influence operations receives significantly higher user engagement compared to influence operations from any other country in our dataset. Our code is available at https://github.com/shafin191/Toxic_IO.