Which side are you on? Insider-Outsider classification in conspiracy-theoretic social media

TL;DR

This paper introduces a new NLP task of classifying insiders and outsiders in conspiracy-related social media posts, addressing complex contextual and implicit cues, and provides a labeled dataset and a novel classification model.

Contribution

The paper presents the first dataset (CT5K) and a specialized model (NP2IO) for insider-outsider classification in conspiracy social media content.

Findings

NP2IO outperforms baseline models by 20%

Model generalizes to unseen noun phrases

Addresses complex contextual and implicit cues

Abstract

Social media is a breeding ground for threat narratives and related conspiracy theories. In these, an outside group threatens the integrity of an inside group, leading to the emergence of sharply defined group identities: Insiders -- agents with whom the authors identify and Outsiders -- agents who threaten the insiders. Inferring the members of these groups constitutes a challenging new NLP task: (i) Information is distributed over many poorly-constructed posts; (ii) Threats and threat agents are highly contextual, with the same post potentially having multiple agents assigned to membership in either group; (iii) An agent's identity is often implicit and transitive; and (iv) Phrases used to imply Outsider status often do not follow common negative sentiment patterns. To address these challenges, we define a novel Insider-Outsider classification task. Because we are not aware of any appropriate existing datasets or attendant models, we introduce a labeled dataset (CT5K) and design a model (NP2IO) to address this task. NP2IO leverages pretrained language modeling to classify Insiders and Outsiders. NP2IO is shown to be robust, generalizing to noun phrases not seen during training, and exceeding the performance of non-trivial baseline models by $20\%$.