Down the Rabbit Hole: Detecting Online Extremism, Radicalisation, and Politicised Hate Speech

TL;DR

This paper reviews and synthesizes research on detecting online extremism, radicalisation, and hate speech, emphasizing the importance of responsible AI and providing guidelines for future research and policy.

Contribution

It offers the first comprehensive systematic review of ERH detection methods, addressing biases and proposing a unified research framework and practical guidelines.

Findings

Textual transformer-based algorithms outperform other models.

Identified biases in ERH research, especially in Oceania/Australasia.

Provided a roadmap for safer online content moderation.

Abstract

Social media is a modern person's digital voice to project and engage with new ideas and mobilise communities $\unicode{x2013}$ a power shared with extremists. Given the societal risks of unvetted content-moderating algorithms for Extremism, Radicalisation, and Hate speech (ERH) detection, responsible software engineering must understand the who, what, when, where, and why such models are necessary to protect user safety and free expression. Hence, we propose and examine the unique research field of ERH context mining to unify disjoint studies. Specifically, we evaluate the start-to-finish design process from socio-technical definition-building and dataset collection strategies to technical algorithm design and performance. Our 2015-2021 51-study Systematic Literature Review (SLR) provides the first cross-examination of textual, network, and visual approaches to detecting extremist affiliation, hateful content, and radicalisation towards groups and movements. We identify consensus-driven ERH definitions and propose solutions to existing ideological and geographic biases, particularly due to the lack of research in Oceania/Australasia. Our hybridised investigation on Natural Language Processing, Community Detection, and visual-text models demonstrates the dominating performance of textual transformer-based algorithms. We conclude with vital recommendations for ERH context mining researchers and propose an uptake roadmap with guidelines for researchers, industries, and governments to enable a safer cyberspace.