A Bootstrapped Model to Detect Abuse and Intent in White Supremacist Corpora

TL;DR

This paper introduces a bootstrapped deep learning model that detects intent and abuse in white supremacist texts, helping distinguish between harmful rhetoric and actual violent plans, validated against crowd-sourced labels.

Contribution

It presents a novel bootstrapped approach combining n-gram and attention-based models to identify intent in extremist language, improving detection accuracy.

Findings

Models converge to stable predictions in few rounds

Merged intent and abuse detection effectively identifies violent posts

Validated predictions align well with crowd-sourced labels

Abstract

Intelligence analysts face a difficult problem: distinguishing extremist rhetoric from potential extremist violence. Many are content to express abuse against some target group, but only a few indicate a willingness to engage in violence. We address this problem by building a predictive model for intent, bootstrapping from a seed set of intent words, and language templates expressing intent. We design both an n-gram and attention-based deep learner for intent and use them as colearners to improve both the basis for prediction and the predictions themselves. They converge to stable predictions in a few rounds. We merge predictions of intent with predictions of abusive language to detect posts that indicate a desire for violent action. We validate the predictions by comparing them to crowd-sourced labelling. The methodology can be applied to other linguistic properties for which a plausible starting point can be defined.