Modelling the Spread of Toxicity and Exploring its Mitigation on Online Social Networks

TL;DR

This paper models how toxicity spreads on social networks by considering user responses as transformations, and demonstrates that targeted interventions like peace-bots can effectively reduce toxicity depending on network structure.

Contribution

It introduces a novel user transformation-based model of toxicity spread and proposes an intervention strategy using peace-bots, advancing beyond traditional epidemic models.

Findings

Toxicity is not conserved in social networks.

Only a subset of users change behavior over time.

Peace-bots can effectively reduce toxicity depending on placement.

Abstract

Hate speech on online platforms has been credibly linked to multiple instances of real world violence. This calls for an urgent need to understand how toxic content spreads and how it might be mitigated on online social networks, and expectedly has been the topic of extensive research in recent times. Prior work has largely modelled hate through epidemic or spread activation based diffusion models, in which the users are often divided into two categories, hateful or not. In this work, users are treated as transformers of toxicity, based on how they respond to incoming toxicity. Compared with the incoming toxicity, users amplify, attenuate, or replicate (effectively, transform) the toxicity and send it forward. We do a temporal analysis of toxicity on Twitter, Koo and Gab and find that (a) toxicity is not conserved in the network; (b) only a subset of users change behaviour over time; and (c) there is no evidence of homophily among behaviour-changing users. In our model, each user transforms incoming toxicity by applying a "shift" to it prior to sending it forward. Based on this, we develop a network model of toxicity spread that incorporates time-varying behaviour of users. We find that the "shift" applied by a user is dependent on the input toxicity and the category. Based on this finding, we propose an intervention strategy for toxicity reduction. This is simulated by deploying peace-bots. Through experiments on both real-world and synthetic networks, we demonstrate that peace-bot interventions can reduce toxicity, though their effectiveness depends on network structure and placement strategy.