Modeling Profanity and Hate Speech in Social Media with Semantic Subspaces

TL;DR

This paper introduces a method to identify and utilize profane subspaces in word and sentence embeddings, improving zero-shot hate speech detection across multiple languages and demonstrating significant transferability over standard models.

Contribution

The study presents a novel approach to model profanity and hate speech using semantic subspaces, enhancing zero-shot transferability across languages and tasks.

Findings

Subspace-based representations transfer more effectively than BERT in zero-shot settings.

Improvements in F1 scores ranged from +10.9 to +42.9 across languages and tasks.

Effective cross-lingual generalization for hate speech detection.

Abstract

Hate speech and profanity detection suffer from data sparsity, especially for languages other than English, due to the subjective nature of the tasks and the resulting annotation incompatibility of existing corpora. In this study, we identify profane subspaces in word and sentence representations and explore their generalization capability on a variety of similar and distant target tasks in a zero-shot setting. This is done monolingually (German) and cross-lingually to closely-related (English), distantly-related (French) and non-related (Arabic) tasks. We observe that, on both similar and distant target tasks and across all languages, the subspace-based representations transfer more effectively than standard BERT representations in the zero-shot setting, with improvements between F1 +10.9 and F1 +42.9 over the baselines across all tested monolingual and cross-lingual scenarios.