Combating high variance in Data-Scarce Implicit Hate Speech Classification

TL;DR

This paper addresses the challenge of high variance in implicit hate speech classification with limited data by exploring optimization techniques and proposing a novel RoBERTa-based model that achieves state-of-the-art results.

Contribution

It introduces a new RoBERTa-based model combined with optimization and regularization methods specifically designed for implicit hate speech detection in data-scarce scenarios.

Findings

Achieved state-of-the-art performance on implicit hate speech datasets.

Demonstrated effectiveness of optimization and regularization techniques in high variance settings.

Improved robustness of hate speech classification models with limited data.

Abstract

Hate speech classification has been a long-standing problem in natural language processing. However, even though there are numerous hate speech detection methods, they usually overlook a lot of hateful statements due to them being implicit in nature. Developing datasets to aid in the task of implicit hate speech classification comes with its own challenges; difficulties are nuances in language, varying definitions of what constitutes hate speech, and the labor-intensive process of annotating such data. This had led to a scarcity of data available to train and test such systems, which gives rise to high variance problems when parameter-heavy transformer-based models are used to address the problem. In this paper, we explore various optimization and regularization techniques and develop a novel RoBERTa-based model that achieves state-of-the-art performance.