Towards countering hate speech against journalists on social media

TL;DR

This paper presents a multilingual Twitter dataset focused on hate speech against journalists, developed through active learning, and evaluates deep learning models, including an ensemble, for improved hate speech detection.

Contribution

It introduces a new multilingual hate speech dataset targeting journalists and proposes an ensemble model that outperforms individual classifiers.

Findings

Ensemble model achieves higher accuracy than single models.

Annotated dataset covers five languages, enhancing multilingual research.

Deep learning architectures effectively detect hate speech against journalists.

Abstract

The damaging effects of hate speech on social media are evident during the last few years, and several organizations, researchers and social media platforms tried to harness them in various ways. Despite these efforts, social media users are still affected by hate speech. The problem is even more apparent to social groups that promote public discourse, such as journalists. In this work, we focus on countering hate speech that is targeted to journalistic social media accounts. To accomplish this, a group of journalists assembled a definition of hate speech, taking into account the journalistic point of view and the types of hate speech that are usually targeted against journalists. We then compile a large pool of tweets referring to journalism-related accounts in multiple languages. In order to annotate the pool of unlabeled tweets according to the definition, we follow a concise annotation strategy that involves active learning annotation stages. The outcome of this paper is a novel, publicly available collection of Twitter datasets in five different languages. Additionally, we experiment with state-of-the-art deep learning architectures for hate speech detection and use our annotated datasets to train and evaluate them. Finally, we propose an ensemble detection model that outperforms all individual models.