It's a Thin Line Between Love and Hate: Using the Echo in Modeling Dynamics of Racist Online Communities

TL;DR

This paper develops a multi-modal neural approach combining language and network analysis to detect and understand racist online communities, revealing their leaders, intersections of hate, and influence of external actors like IRA trolls.

Contribution

It introduces a novel dataset and a multi-modal neural architecture that outperforms baselines in identifying hate community leaders and analyzing hate speech dynamics.

Findings

Multi-modal model effectively detects hate community leaders.

Language and network features reveal intersectionality of hate.

IRA trolls play a significant role in hate network dynamics.

Abstract

The (((echo))) symbol -- triple parenthesis surrounding a name, made it to mainstream social networks in early 2016, with the intensification of the U.S. Presidential race. It was used by members of the alt-right, white supremacists and internet trolls to tag people of Jewish heritage -- a modern incarnation of the infamous yellow badge (Judenstern) used in Nazi-Germany. Tracking this trending meme, its meaning, and its function has proved elusive for its semantic ambiguity (e.g., a symbol for a virtual hug). In this paper we report of the construction of an appropriate dataset allowing the reconstruction of networks of racist communities and the way they are embedded in the broader community. We combine natural language processing and structural network analysis to study communities promoting hate. In order to overcome dog-whistling and linguistic ambiguity, we propose a multi-modal neural architecture based on a BERT transformer and a BiLSTM network on the tweet level, while also taking into account the users ego-network and meta features. Our multi-modal neural architecture outperforms a set of strong baselines. We further show how the the use of language and network structure in tandem allows the detection of the leaders of the hate communities. We further study the ``intersectionality'' of hate and show that the antisemitic echo correlates with hate speech that targets other minority and protected groups. Finally, we analyze the role IRA trolls assumed in this network as part of the Russian interference campaign. Our findings allow a better understanding of recent manifestations of racism and the dynamics that facilitate it.