Exploring multi-task multi-lingual learning of transformer models for hate speech and offensive speech identification in social media

TL;DR

This paper explores multi-task, multi-lingual transformer models for hate speech detection across languages, demonstrating improved generalization and efficiency through combined training approaches, and providing open-source code for further research.

Contribution

It introduces multi-task, multi-lingual transformer models trained with three approaches, enhancing hate speech detection across languages with reduced inference costs.

Findings

Multi-task models improve cross-language generalization.

Combined training approaches balance accuracy and efficiency.

Open-source code facilitates further research.

Abstract

Hate Speech has become a major content moderation issue for online social media platforms. Given the volume and velocity of online content production, it is impossible to manually moderate hate speech related content on any platform. In this paper we utilize a multi-task and multi-lingual approach based on recently proposed Transformer Neural Networks to solve three sub-tasks for hate speech. These sub-tasks were part of the 2019 shared task on hate speech and offensive content (HASOC) identification in Indo-European languages. We expand on our submission to that competition by utilizing multi-task models which are trained using three approaches, a) multi-task learning with separate task heads, b) back-translation, and c) multi-lingual training. Finally, we investigate the performance of various models and identify instances where the Transformer based models perform differently and better. We show that it is possible to to utilize different combined approaches to obtain models that can generalize easily on different languages and tasks, while trading off slight accuracy (in some cases) for a much reduced inference time compute cost. We open source an updated version of our HASOC 2019 code with the new improvements at https://github.com/socialmediaie/MTML_HateSpeech.