NLPineers@ NLU of Devanagari Script Languages 2025: Hate Speech Detection using Ensembling of BERT-based models

TL;DR

This paper develops an ensemble of BERT-based models for hate speech detection in Hindi and Nepali, achieving competitive recall and F1 scores, and introduces data augmentation techniques to handle class imbalance.

Contribution

It introduces an ensemble approach of multilingual BERT models for hate speech detection in Devanagari languages, with novel data augmentation methods to improve performance.

Findings

Achieved recall of 0.7762 and F1 score of 0.6914.

Ensemble models outperformed individual models in detection accuracy.

Data augmentation via backtranslation improved class balance.

Abstract

This paper explores hate speech detection in Devanagari-scripted languages, focusing on Hindi and Nepali, for Subtask B of the CHIPSAL@COLING 2025 Shared Task. Using a range of transformer-based models such as XLM-RoBERTa, MURIL, and IndicBERT, we examine their effectiveness in navigating the nuanced boundary between hate speech and free expression. Our best performing model, implemented as ensemble of multilingual BERT models achieve Recall of 0.7762 (Rank 3/31 in terms of recall) and F1 score of 0.6914 (Rank 17/31). To address class imbalance, we used backtranslation for data augmentation, and cosine similarity to preserve label consistency after augmentation. This work emphasizes the need for hate speech detection in Devanagari-scripted languages and presents a foundation for further research.