Noisy Self-Training with Data Augmentations for Offensive and Hate Speech Detection Tasks

TL;DR

This paper investigates the effectiveness of self-training and noisy self-training with data augmentation techniques for offensive and hate speech detection, finding that while self-training improves performance, noisy approaches may decrease it.

Contribution

It provides a comprehensive evaluation of default and noisy self-training methods with various data augmentations across multiple BERT models for hate speech detection.

Findings

Self-training improves F1-macro scores by up to 1.5%.

Noisy self-training with augmentations decreases performance.

Performance gains are consistent across different model sizes.

Abstract

Online social media is rife with offensive and hateful comments, prompting the need for their automatic detection given the sheer amount of posts created every second. Creating high-quality human-labelled datasets for this task is difficult and costly, especially because non-offensive posts are significantly more frequent than offensive ones. However, unlabelled data is abundant, easier, and cheaper to obtain. In this scenario, self-training methods, using weakly-labelled examples to increase the amount of training data, can be employed. Recent "noisy" self-training approaches incorporate data augmentation techniques to ensure prediction consistency and increase robustness against noisy data and adversarial attacks. In this paper, we experiment with default and noisy self-training using three different textual data augmentation techniques across five different pre-trained BERT architectures varying in size. We evaluate our experiments on two offensive/hate-speech datasets and demonstrate that (i) self-training consistently improves performance regardless of model size, resulting in up to +1.5% F1-macro on both datasets, and (ii) noisy self-training with textual data augmentations, despite being successfully applied in similar settings, decreases performance on offensive and hate-speech domains when compared to the default method, even with state-of-the-art augmentations such as backtranslation.