# Advancing cyberbullying detection in low-resource languages: a transformer- stacking framework for Bengali

## TL;DR

This paper introduces a new framework for detecting cyberbullying in Bengali, a low-resource language, using stacked transformer models and achieves high accuracy.

## Contribution

The novel contribution is the Transformer-stacking framework with Bengali-specific preprocessing for improved cyberbullying detection.

## Key findings

- Transformer-stacking achieved 93.61% F1-score and 93.62% accuracy for binary cyberbullying classification.
- The framework outperformed eight baseline models and recent state-of-the-art methods.
- It demonstrated scalability and adaptability on external Bengali datasets for hate speech and abusive language.

## Abstract

Cyberbullying on social networks has emerged as a pressing global issue, yet research in low-resource languages such as Bengali remains underdeveloped due to the scarcity of high-quality datasets, linguistic resources, and targeted methodologies. Many existing approaches overlook essential language-specific preprocessing, neglect the integration of advanced transformer-based models, and do not adequately address model validation, scalability, and adaptability. To address these limitations, this study introduces three Bengali-specific preprocessing strategies to enhance feature representation. It then proposes Transformer-stacking, an effective hybrid detection framework that combines three transformer models, XLM-R-base, multilingual BERT, and Bangla-Bert-Base, via a stacking strategy with a multi-layer perceptron classifier. The framework is evaluated on a publicly available Bengali cyberbullying dataset comprising 44,001 samples across both binary (Sub-task A) and multiclass (Sub-task B) classification settings. Transformer-stacking achieves an F1-score of 93.61% and an accuracy of 93.62% for Sub-task A, and an F1-score and accuracy of 89.23% for Sub-task B, outperforming eight baseline transformer models, four transformer ensemble techniques, and recent state-of-the-art methods. These improvements are statistically validated using McNemar's test. Furthermore, experiments on two external Bengali datasets, focused on hate speech and abusive language, demonstrate the model's scalability and adaptability. Overall, Transformer-stacking offers an effective and generalizable solution for Bengali cyberbullying detection, establishing a new benchmark in this underexplored domain.