An Online Multilingual Hate speech Recognition System

TL;DR

This paper presents a multilingual hate speech recognition system that combines multiple datasets, improves classification performance, and deploys a real-time scoring tool, demonstrating effectiveness in English and Hindi languages.

Contribution

It introduces a unified dataset approach, optimizes a hate speech classifier, and develops a real-time feedback tool for multilingual hate speech detection.

Findings

Achieved competitive performance on English and Hindi datasets.

Developed a real-time scoring and feedback system.

Outperformed many monolingual models in multilingual settings.

Abstract

The exponential increase in the use of the Internet and social media over the last two decades has changed human interaction. This has led to many positive outcomes, but at the same time it has brought risks and harms. While the volume of harmful content online, such as hate speech, is not manageable by humans, interest in the academic community to investigate automated means for hate speech detection has increased. In this study, we analyse six publicly available datasets by combining them into a single homogeneous dataset and classify them into three classes, abusive, hateful or neither. We create a baseline model and we improve model performance scores using various optimisation techniques. After attaining a competitive performance score, we create a tool which identifies and scores a page with effective metric in near-real time and uses the same as feedback to re-train our model. We prove the competitive performance of our multilingual model on two langauges, English and Hindi, leading to comparable or superior performance to most monolingual models.