Comparison of Modern Multilingual Text Embedding Techniques for Hate Speech Detection Task

TL;DR

This study evaluates modern multilingual sentence embeddings for hate speech detection across Lithuanian, Russian, and English, introducing a new Lithuanian hate speech corpus and benchmarking six encoders with various models.

Contribution

It introduces LtHate, a Lithuanian hate speech dataset, and benchmarks six multilingual encoders with supervised and unsupervised models, analyzing the impact of feature dimensionality and modeling choices.

Findings

Supervised models outperform anomaly detection in hate speech classification.

Best configurations achieve up to 80.96% accuracy and 0.887 AUC ROC in Lithuanian.

PCA compression retains most discriminative power in supervised models.

Abstract

Online hate speech and abusive language pose a growing challenge for content moderation, especially in multilingual settings and for low-resource languages such as Lithuanian. This paper investigates to what extent modern multilingual sentence embedding models can support accurate hate speech detection in Lithuanian, Russian, and English, and how their performance depends on downstream modeling choices and feature dimensionality. We introduce LtHate, a new Lithuanian hate speech corpus derived from news portals and social networks, and benchmark six modern multilingual encoders (potion, gemma, bge, snow, jina, e5) on LtHate, RuToxic, and EnSuperset using a unified Python pipeline. For each embedding, we train both a one class HBOS anomaly detector and a two class CatBoost classifier, with and without principal component analysis (PCA) compression to 64-dimensional feature vectors. Across all datasets, two class supervised models consistently and substantially outperform one class anomaly detection, with the best configurations achieving up to 80.96% accuracy and AUC ROC of 0.887 in Lithuanian (jina), 92.19% accuracy and AUC ROC of 0.978 in Russian (e5), and 77.21% accuracy and AUC ROC of 0.859 in English (e5 with PCA). PCA compression preserves almost all discriminative power in the supervised setting, while showing some negative impact for the unsupervised anomaly detection case. These results demonstrate how modern multilingual sentence embeddings combined with gradient boosted decision trees provide robust soft-computing solutions for multilingual hate speech detection applications.